Robots.txt is not an appropriate or effective way of blocking sensitive or confidential material. It only instructs well-behaved crawlers that the pages are not for them, but it does not prevent your server from delivering those pages to a browser that requests them. One reason is that search engines could still reference the URLs you block (showing just the URL, no title or snippet) if there happen to be links to those URLs somewhere on the Internet (like referrer logs). Also, non-compliant or rogue search engines that don't acknowledge the Robots Exclusion Standard could disobey the instructions of your robots.txt. Finally, a curious user could examine the directories or subdirectories in your robots.txt file and guess the URL of the content that you don't want seen.
To prevent users from linking to one version of a URL and others linking to a different version (this could split the reputation of that content between the URLs), focus on using and referring to one URL in the structure and internal linking of your pages. If you do find that people are accessing the same content through multiple URLs, setting up a 301 redirect32 from non-preferred URLs to the dominant URL is a good solution for this. You may also use canonical URL or use the rel="canonical"33 link element if you cannot redirect.
In Chapter 2, we learned about SERP features. That background is going to help us understand how searchers want to consume information for a particular keyword. The format in which Google chooses to display search results depends on intent, and every query has a unique one. Google describes these intents in their Quality Rater Guidelines as either “know” (find information), “do” (accomplish a goal), “website” (find a specific website), or “visit-in-person” (visit a local business).
QUOTE: “The purpose of a page is the reason or reasons why the page was created. Every page on the Internet is created for a purpose, or for multiple purposes. Most pages are created to be helpful for users, thus having a beneficial purpose. Some pages are created merely to make money, with little or no effort to help users. Some pages are even created to cause harm to users. The first step in understanding a page is figuring out its purpose.” Google Search Quality Evaluator Guidelines 2019
Basically, SEO keyword research should be an ongoing and ever-evolving part of your job as a marketer. Old keywords need to be reevaluated periodically, and high-volume, competitive keywords (or “head” keywords, as opposed to long-tailed keywords) can often be usefully replaced or augmented with longer, more specific phrases designed not to bring in just any visitor but exactly the right visitors. (Who visits your site – particularly if they’re people who are actively looking for your services – is at least as important as how many people visit.)
Email Marketing: Email can be an effective way to maintain a connection between your business and its customers. You can purchase email addresses of customers and prospective customers, but the best results usually come from emails collected on your website. You can entice people to give you their emails through a quality free offer, such as a downloadable resource, called a lead magnet. Once you have their email, you can send a newsletter, special offers, and other information your target market would be interested in—as long as you follow laws and regulations around email marketing.

Be sure to re-evaluate these keywords every few months -- once a quarter is a good benchmark, but some businesses like to do it even more often than that. As you gain even more authority in the SERPs, you'll find that you can add more and more keywords to your lists to tackle as you work on maintaining your current presence, and then growing in new areas on top of that.


QUOTE: “When our quality algorithms go to your website, and they see that there’s some good content here on this page, but there’s some really bad or kind of low quality content on the bottom part of the page, then we kind of have to make a judgment call on these pages themselves and say, well, some good, some bad. Is this overwhelmingly bad? Is this overwhelmingly good? Where do we draw the line?” John Mueller, Google 2016
Ranking refers to the process search engines use to determine where a particular piece of content should appear on a SERP. Search visibility refers to how prominently a piece of content is displayed in search engine results. Highly visible content (usually the content that ranks highest) may appear right at the top of organic search results or even in a featured snippet, while less-visible content may not appear until searchers click to page two and beyond
Understanding the balance of terms that might be a little more difficult due to competition, versus those terms that are a little more realistic, will help you maintain a similar balance that the mix of long-tail and head terms allows. Remember, the goal is to end up with a list of keywords that provide some quick wins but also helps you make progress toward bigger, more challenging SEO goals.
Use the Keyword Planner to flag any terms on your list that have way too little (or way too much) search volume, and don't help you maintain a healthy mix like we talked about above. But before you delete anything, check out their trend history and projections in Google Trends. You can see whether, say, some low-volume terms might actually be something you should invest in now -- and reap the benefits for later.
Search engine marketing (SEM) is related to SEO in that they both deal with digital marketing outreach on search engines like Google and Bing. However, SEO typically refers to internal, organic website optimization while search engine marketing is commonly known as advertising through a paid media budget. While SEO will help your website traffic increase through natural algorithmic means, SEM earns traffic through the process of purchasing ads on search engines. In the image to the left, you will see a search engine results page that is the result of typing the keyword “digital marketing” into Google. The top four results have a small green box that says “ad” underneath the primary title link. This signifies paid advertising, or search engine marketing, where a brand or business has paid Google to display their ads at the top of the page for that specific keyword.
Robots.txt is not an appropriate or effective way of blocking sensitive or confidential material. It only instructs well-behaved crawlers that the pages are not for them, but it does not prevent your server from delivering those pages to a browser that requests them. One reason is that search engines could still reference the URLs you block (showing just the URL, no title or snippet) if there happen to be links to those URLs somewhere on the Internet (like referrer logs). Also, non-compliant or rogue search engines that don't acknowledge the Robots Exclusion Standard could disobey the instructions of your robots.txt. Finally, a curious user could examine the directories or subdirectories in your robots.txt file and guess the URL of the content that you don't want seen.
Consider the average cost-per-click in your industry. Before deciding that SEM is right for your business, research and consider how much you’ll need to spend to show in paid search results. Keywords have varying cost-per-clicks based on competition. If your cost-per-click is low, it might be the right strategy for you. On the flipside, a very high cost-per-click might make you decide you’re better off focusing on SEO.

SEM is better for testing than SEO. Because you can immediately turn SEM paid ads off and on, it’s a great strategy for testing. You can quickly revise your ad copy, target new audiences, and change landing page content to test your new tactics. This flexibility allows you to see differences in your strategies immediately. You cannot accomplish this through SEO, as it would take too much time to make changes and monitor differences in results.

I prefer simple SEO techniques and ones that can be measured in some way. I have never just wanted to rank for competitive terms; I have always wanted to understand at least some of the reasons why a page ranked for these key phrases. I try to create a good user experience for humans AND search engines. If you make high-quality text content relevant and suitable for both these audiences, you’ll more than likely find success in organic listings and you might not ever need to get into the technical side of things, like redirects and search engine friendly URLs.
Tablet - We consider tablets as devices in their own class, so when we speak of mobile devices, we generally do not include tablets in the definition. Tablets tend to have larger screens, which means that, unless you offer tablet-optimized content, you can assume that users expect to see your site as it would look on a desktop browser rather than on a smartphone browser.
QUOTE: “To summarize, a lack of helpful SC may be a reason for a Low quality rating, depending on the purpose of the page and the type of website. We have different standards for small websites which exist to serve their communities versus large websites with a large volume of webpages and content. For some types of “webpages,” such as PDFs and JPEG files, we expect no SC at all.” Google Search Quality Evaluator Guidelines 2015
Additionally, there are many situations where PPC (a component of SEM) makes more sense than SEO. For example, if you are first launching a site and you want immediate visibility, it is a good idea to create a PPC campaign because it takes less time than SEO, but it would be unwise to strictly work with PPC and not even touch search engine optimization.
Google asks quality raters to investigate your reputation by searching “giving the example [“ibm.com” reviews –site:ibm.com]: A search on Google for reviews of “ibm.com” which excludes pages on ibm.com.” – So I would do that search yourself and judge for yourself what your reputation is. Very low ratings on independent websites could play a factor in where you rank in the future – ” with Google stating clearly “very low ratings on the BBB site to be evidence for a negative reputation“. Other sites mentioned to review your business include YELP and Amazon. Often – using rich snippets containing schema.org information – you can get Google to display user ratings in the actual SERPs. I noted you can get ‘stars in SERPs’ within two days after I added the code (March 2014).
QUOTE: “… it also includes things like the comments, includes the things like the unique and original content that you’re putting out on your site that is being added through user-generated content, all of that as well. So while I don’t really know exactly what our algorithms are looking at specifically with regards to your website, it’s something where sometimes you go through the articles and say well there is some useful information in this article that you’re sharing here, but there’s just lots of other stuff happening on the bottom of these blog posts. When our algorithms look at these pages, in an aggregated way across the whole page, then that’s something where they might say well, this is a lot of content that is unique to this page, but it’s not really high quality content that we want to promote in a very visible way. That’s something where I could imagine that maybe there’s something you could do, otherwise it’s really tricky I guess to look at specific changes you can do when it comes to our quality algorithms.” John Mueller, Google 2016
Wow! This was so helpful to me. I am new to the blogging world and have been feeling really frustrated and discouraged because I lacked the knowledge of getting my post to rank in search engines. I know there is a lot more I still need to learn but this has layed a foundation for me. I am bookmarking it so I can return and read it again. Thank you for writing!
SEM search results have ad extensions. SEO search results have featured snippets. When comparing SEM vs. SEO, you’ll also find differences in the appearance of the search results. SEM search results may include ad extensions, which can add on additional links, phone numbers, and callouts. On the other hand, SEO results may appear with featured snippets in search.
QUOTE: “News and current events: news about important topics such as international events, business, politics, science, technology, etc. Keep in mind that not all news articles are necessarily considered YMYL (e.g., sports, entertainment, and everyday lifestyle topics are generally not YMYL). Please use your judgment and knowledge of your locale. ● Civics, government, and law: information important to maintaining an informed citizenry, such as information about voting, government agencies, public institutions, social services, and legal issues (e.g., divorce, child custody, adoption, creating a will, etc.). ● Finance: financial advice or information regarding investments, taxes, retirement planning, loans, banking, or insurance, particularly webpages that allow people to make purchases or transfer money online. ● Shopping: information about or services related to research or purchase of goods/services, particularly webpages that allow people to make purchases online. ● Health and safety: advice or information about medical issues, drugs, hospitals, emergency preparedness, how dangerous an activity is, etc. ● Groups of people: information about or claims related to groups of people, including but not limited to those grouped on the basis of race or ethnic origin, religion, disability, age, nationality, veteran status, sexual orientation, gender or gender identity. ● Other: there are many other topics related to big decisions or important aspects of people’s lives which thus may be considered YMYL, such as fitness and nutrition, housing information, choosing a college, finding a job, etc.” Google Search Quality Evaluator Guidelines 2019
Many search engine marketers think who you link out to (and who links to you) helps determine a topical community of sites in any field or a hub of authority. Quite simply, you want to be in that hub, at the centre if possible (however unlikely), but at least in it. I like to think of this one as a good thing to remember in the future as search engines get even better at determining topical relevancy of pages, but I have never actually seen any granular ranking benefit (for the page in question) from linking out.
QUOTE: “Medium pages achieve their purpose and have neither high nor low expertise, authoritativeness, and trustworthiness. However, Medium pages lack the characteristics that would support a higher quality rating. Occasionally, you will find a page with a mix of high and low quality characteristics. In those cases, the best page quality rating may be Medium.” Google Quality Evaluator Guidelines, 2017
But essentially the idea there is that this is a good representative of the the content from your website and that’s all that we would show to users on the other hand if someone is specifically looking for let’s say dental bridges in Dublin then we’d be able to show the appropriate clinic that you have on your website that matches that a little bit better so we’d know dental bridges is something that you have a lot on your website and Dublin is something that’s unique to this specific page so we’d be able to pull that out and to show that to the user like that so from a pure content duplication point of view that’s not really something I totally worry about.
The higher the search volume for a given keyword or keyword phrase, the more work is typically required to achieve higher rankings. This is often referred to as keyword difficulty and occasionally incorporates SERP features; for example, if many SERP features (like featured snippets, knowledge graph, carousels, etc) are clogging up a keyword’s result page, difficulty will increase. Big brands often take up the top 10 results for high-volume keywords, so if you’re just starting out on the web and going after the same keywords, the uphill battle for ranking can take years of effort.
While the obvious purpose of internet marketing is to sell goods, services or advertising over the internet, it's not the only purpose a business using internet marketing may have; a company may be marketing online to communicate a message about itself (building its brand) or to conduct research. Online marketing can be a very effective way to identify a target market or discover a marketing segment's wants and needs. (Learn more about conducting market research).
The higher the search volume for a given keyword or keyword phrase, the more work is typically required to achieve higher rankings. This is often referred to as keyword difficulty and occasionally incorporates SERP features; for example, if many SERP features (like featured snippets, knowledge graph, carousels, etc) are clogging up a keyword’s result page, difficulty will increase. Big brands often take up the top 10 results for high-volume keywords, so if you’re just starting out on the web and going after the same keywords, the uphill battle for ranking can take years of effort.
If you are just starting out, don’t think you can fool Google about everything all the time. Google has VERY probably seen your tactics before. So, it’s best to keep your plan simple. GET RELEVANT. GET REPUTABLE. Aim for a healthy, satisfying visitor experience. If you are just starting out – you may as well learn how to do it within Google’s Webmaster Guidelines first. Make a decision, early, if you are going to follow Google’s guidelines, or not, and stick to it. Don’t be caught in the middle with an important project. Do not always follow the herd.
I’ve got by, by thinking external links to other sites should probably be on single pages deeper in your site architecture, with the pages receiving all your Google Juice once it’s been “soaked up” by the higher pages in your site structure (the home page, your category pages). This tactic is old school but I still follow it. I don’t need to think you need to worry about that, too much, in 2020.

QUOTE: “The purpose of a page is the reason or reasons why the page was created. Every page on the Internet is created for a purpose, or for multiple purposes. Most pages are created to be helpful for users, thus having a beneficial purpose. Some pages are created merely to make money, with little or no effort to help users. Some pages are even created to cause harm to users. The first step in understanding a page is figuring out its purpose.” Google Search Quality Evaluator Guidelines 2019


QUOTE: “In place of a pop-up try a full-screen inline ad. It offers the same amount of screen real estate as pop-ups without covering up any content. Fixing the problem depends on the issue you have for example if it’s a pop-up you’ll need to remove all the pop-up ads from your site but if the issue is high ad density on a page you’ll need to reduce the number of ads” Google, 2017

At the moment, I don’t know you, your business, your website, your resources, your competition or your product. Even with all that knowledge, calculating ROI is extremely difficult because ultimately Google decides on who ranks where in its results – sometimes that’s ranking better sites, and sometimes (often) it is ranking sites breaking the rules above yours.
QUOTE: “For instance, we would see a lot of low-quality posts in a forum. We would index those low-quality pages. And we’d also see a lot of really high-quality posts, with good discussions, good information on those pages. And our algorithms would be kind of stuck in a situation with, well, there’s a lot of low-quality content here, but there’s also a lot of high-quality content here. So how should we evaluate the site overall? And usually, what happens is, our algorithms kind of find some middle ground……. what you’d need to do to, kind of, move a step forward, is really try to find a way to analyze the quality of your content, and to make sure that the high-quality content is indexed and that the lower-quality content doesn’t get indexed by default.” John Mueller, Google 2014
Let’s look at an example. At Yoast, we call our courses platform the “Yoast Academy.” So at first, it might seem logical for us to optimize for the keyword “Yoast Academy.” However, when we analyze traffic data, it turns out that our audience searches for “Yoast courses” way more often. So it makes much more sense to optimize for that term instead. Every company has its internal vocabulary, which often doesn’t match the vocabulary of its audience. Therefore, you should always select your keywords from your audience’s perspective. You can use Google Trends to research how often certain search terms are used compared to other terms.
Search engine optimization is a method for sustainably influences search engine rankings. Google and other search engines calculate their search results for keywords using highly complex algorithms. The individual ranking factors and their weighting within the ranking calculation are well-guarded intellectual property that belongs to the search engines and is not publicly disclosed.

Think about how Google can algorithmically and manually determine the commercial intent of your website – think about the signals that differentiate a real small business website from a website created JUST to send visitors to another website with affiliate links, on every page, for instance; or adverts on your site, above the fold, etc, can be a clear indicator of a webmaster’s particular commercial intent – hence why Google has a Top Heavy Algorithm.
If you're a regular blogger, these are probably the topics you blog about most frequently. Or perhaps they're the topics that come up the most in sales conversations. Put yourself in the shoes of your buyer personas -- what types of topics would your target audience search that you'd want your business to get found for? If you were a company like HubSpot, for example -- selling marketing software (which happens to have some awesome SEO tools ... but I digress ... you might have general topic buckets like:

A navigational page is a simple page on your site that displays the structure of your website, and usually consists of a hierarchical listing of the pages on your site. Visitors may visit this page if they are having problems finding pages on your site. While search engines will also visit this page, getting good crawl coverage of the pages on your site, it's mainly aimed at human visitors.
×