Prevent search engines from indexing the text in the sidebar that isn't related to the main content but still allow them to follow the links there

Question

I am developing a CMS in PHP. I have a sidebar on most pages and also some other widgets. For example "read other articles" or a list of categories. Google does index that random-like content as part of the page.

In detail: I have a category for network stuff and a category for programming stuff. Sometimes on a page about networking there is a link in the sidebar to an article about programming. I want search engines, to follow those links but NOT index them as part of the page with the promary content about networking! For example I might write about LDAP but in the sidebar there is something about MBA. When I search Google looking for samba I might find that page.

When I use rel=nofollow, as far as I understand they will not follow those links at all.

So how can I define that some content is not to be indexed as part of a specific URL?

The only possible solution I found so far is

<!--BeginNoIndex-->
<!--EndNoIndex-->

It seems that this tells search engines not to associate content inside those tags with the main page content.

Answer 1

Indexing is not about pieces of content of a web page, but about the entire web page.

As the correct way to treat the issue, you need to wrap the content inside the sidebar in <aside></aside> HTML5 semantic tag which indicates the piece of content inside a web page which isn't directly related to the <main></main> content.

The main part of page needs to be wrapped inside <main> HTML tag. The main text of <main> content needs to be in a nested <article> tag. The related but not the main contents need to be wrapped in <aside> tag, nested in <main> tag, outside the <article> tag.