. . .
zones could be registered and will be indexed. Index framesets and iframes If enabled, both options will index html and image frames. Not available for dynamically reloaded frames (e.g. JavaScript). Follow HTTP redirections Follow URL redirections caused by HTTP 301, 302, 303 and 307 status codes. Also obeying JavaScript, sent as html content
. . .
content is requested by the crawler. Index of RAR and ZIP compressed files and archives Supports compressed (X)html, XML and also PDFs, all kind of feeds, frames and iframes in archives. Links found in the compressed files are followed. Converter included for PDF, DOCX, XLSX, ODT, ODS, CSV, PPTX and XLS files Converting also non-Latin text
. . .
links outside are followed. Multiple and nested divs will be attended. Do not index parts of a page defined by html5 elements <tag> . . . </tag> Foreseen to cooperate with the html5 elements like: section, nav, aside, hgroup, article, header, footer Vice versa function also included in order to index only parts of a page between