By improving different technical elements of your site you can help search engines better find, crawl, and index your site.
Search engines cannot interpret the content of web pages the same way we do. Take this page for example. To you, it is a page containing different visual elements like logos, photos, etc. To an engine it looks like this:
When search engines look at web pages they look at the page HTML code (Hypertext Markup Language). This same code tells your browser where to download all the images from, how things are laid out, what fonts and colors to display etc. The HTML of pages also refer to and load the style sheets (CSS) which are extra instructions for the visual representation of the content.
Keeping a clean and error-free HTML pages help browsers display your pages correctly and help search engines understand the structure of your page.
Much like humans, search engines visit links to find new content. Unlike humans though, engines click on every link they can find. The easiest thing to do to ensure that your content is indexed is to link to it.
Another way to help engines find your content is through XML Sitemaps. A sitemap is a listing of your pages’ content in a format that engines can read through. You can learn more about sitemaps and how to generate them on sitemaps.org. Once you create your sitemaps you submit them directly to the search engines through “webmaster tools”.
Linking and using sitemaps is a great way to tell search engines about the location and changes of your content, however, when there are pages which you do not want to be indexed you can use meta robots. Think of members-only pages you might have or test pages of your site which you might want to hide from being indexed and show on SERPs. To do this you can set rules for search engines in what is called robots.txt file. This file is added to the main root folder of your site. Creating a robots.txt file can be a little technical but you can learn more by visiting the robotstxt.org.
In theory, all pages on the internet should have their unique URL, but in practice however pages often can be reached through slight variations in their URL. This results in duplicate URLs (content) in the search engines index. Having duplicate content on your site is very dangerous because search engines like humans prefer unique content. Duplicate content can cause engines to penalize your site which can really hurt your organic traffic!
One way to resolve duplicate content issues is to use the Rel Canonical Meta Tag. You add this tag to the code of your duplicate pages telling search engines which is the primary URL to access your content. You can also clarify to search engines how you use URL parameters directly in webmaster tools.
Duplicate content issues may also arise when you move content around your site. For example, you may move content from URL A to URL B but both URL A and B can exist in a search engine index. This may cause search engines to send traffic to the wrong page (URL A). To avoid this it is important to use redirects. Here are three of the most important redirects and when to use them:
You can use technologies like JavaScript, Flash, and AJAX to enhance the user experience on your site. Unfortunately, these technologies may cause problems for search engines. The crawlers are consistently becoming more advanced and are getting better all the time, but traditionally they have struggled with these technologies and it is advisable to use them carefully. You can still have this type of element on your site but as a general rule avoid using JavaScript, Flash, and AJAX for your important page elements and content. Site navigation is one example, even the best site architecture in the world can be undermined by navigational elements that are inaccessible to search engines.
Flash is great for making websites look pretty but a bad choice from an SEO point of view. When reading Flash content, search engines have a very difficult time associating specific parts of content with the correct spot in the file. Flash can add a lot of value to a website, but it should be used sparingly. Flash can be implemented in a way that really adds to a website’s user experience, but you should never make a site entirely in Flash (or any significant portion). With the advent and increasing support of HTML5, there are increasingly fewer reasons to use Flash as HTML5 can accomplish most anything you would want to do with Flash. See all those amazing examples of HTML5 sites.
AJAX is an acronym for Asynchronous JavaScript and XML. It is a great technology from a user point of view as it reduces load time and increases site speed by only requesting content when users request it. However, AJAX may cause issues as it requires the search engine to “click” or request the content before it can be read. Refer to the Google Guide to AJAX crawling for webmasters and developers for a detailed list of things you can do to help Google and other Search Engines crawl and index your content.
The hosting and management of your web server can have a tremendous effect on your SERPs ranking. Engines want to provide a good experience to their users so they want to send them to sites that are reliable and have a fast load speed. You need to make sure that you are serving pages quickly and have minimized your server downtime.
The location of your server can have an effect on speed. If most of your users are located half a world away from your server location they may find that your pages are loading slowly. For this reason, you must ensure that your web server is located in the geographical region of the majority of your visitors. If your business is global and you expect traffic from around the world, consider a hosting solution which can distribute a request for your pages to a global network of servers.
Google Search Console and Bing Webmaster Tools let you see what search engines know about your site, whether there were any problems while indexing your pages, as well as other important information. These tools are very useful because they let you provide the engines with directions like uploading sitemaps and setting geo-targeting for your sites. Here are some specific benefits of creating and actively managing webmaster accounts:
Go to google.com/webmaters/tools and sign in/create a google account. When you log in you will be able to add websites you can manage. You will need to verify to Google that you are the webmaster of the site you want to add.
Once you verify your site you will access the dashboard with all the areas of webmaster tools. Detailed tutorials on how to use Google Search Console are available on the Google Search Console YouTube Channel.
Go to bing.com/toolbox/webmaster. Much like Google Webmaster Tools, you can find out what information Bing has about your site, and you can provide instructions. Visit the Bing Webmaster Tools Help & HowTo Center for training and resources.
Keeping a clean code and visible to the engine’s content must be a fundamental part of your SEO efforts. Hopefully, this guide provided you with a technical overview of what you must consider when building and optimizing your websites.