Questions tagged [screaming-frog]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
16 views

Ecommerce site Screaming Frog (and other tools) crawls return internal cart urls with parameters

Example url: somedomaindotcom/patches-metal-rubber-1-can/?add-to-cart=4183 When I crawl my WooCommerce sites with tools like Screaming Frog/AHREFS/SEMRush they return these one-time usage urls that ...
user avatar
1 vote
0 answers
611 views

Screaming Frog reports internal links are unsafe cross-origin

I've just run a Screaming Frog spider against a client's website and it has returned 370 links that are Unsafe Cross-Origin Links. However when I have investigated the links they don't match the ...
user avatar
  • 288
1 vote
0 answers
49 views

Can Screaming Frog check for invalid certificates?

I've been using Screaming Frog to test the home pages of a few hundred sites I manage, to see what status codes they return. I noticed in my browser that one of those sites has an invalid (expired) ...
user avatar
  • 309
0 votes
0 answers
37 views

How Can I Use Screaming Frog to Find Specific Text Using RegEx or XPath?

I'm using Screaming Frog to crawl my site. I created a custom search to look for "myterm". When i crawl the site, the reports shows how many occurrences of "myterm" are shown on ...
user avatar
1 vote
0 answers
23 views

Screaming Frog reporting duplication between two canonicalized pages

I have Page-A with canonical pointing towards page B. Page B should be considered the original and should be ranked in SERPS. But the Screaming Frog crawler is showing that page A and B are ...
user avatar
  • 11
3 votes
1 answer
53 views

Why don't SEO tools such as Screaming frog crawl some of my links?

I have several internal links in my inner pages, for example Interlaken, Switzerland. I used several SEO tools including Screaming Frog to generate sitemap. All SEO tools index links under the "...
user avatar
  • 133
1 vote
1 answer
174 views

Screaming frog crawling, images, JS and CSS files as well

When I plug in my website URL in the free version of Screaming frog, it also lists the images, CSS and JS files. Is it suppose to be like that?
user avatar
1 vote
1 answer
61 views

Is it normal that URL's are getting crawled in Screaming frog Still they have a no-follow and no-index tag on them

I'm getting a lot of URL's in screaming frog where the URLs are no-follow no-index but, they are still getting crawled in screaming frog. Is it fine or should I change anything in this?
user avatar
  • 420
1 vote
1 answer
80 views

Why would Google Search Console report many times more indexed URLs than a Screaming Frog crawl finds?

I've just crawled my website in Screaming Frog and I have around 6K Internal URLs and 4K external URLs. Then I looked in GSC and saw that Google indexed 50K pages. How is that possible?
user avatar
2 votes
1 answer
104 views

How can find I the place where the Screaming Frog found a link on my site?

I use screaming frog for my site. For some URLs it finds I don't know where it found those links. This is an example image of my screaming frog:
user avatar
  • 123
3 votes
1 answer
333 views

Screaming Frog reports "Mailto" link create 404 errors

A site I've built gets too many 404 errors with Screaming Frog All the 404 originate from the "mailto:" links on the site - for some reason the scan identifies non-exist pages which looks like this: ...
user avatar
1 vote
2 answers
60 views

Discrepancy in ScreamingFrog Depth Stats and actual list of URLs

I have crawled my website with Screaming Frog, it has roughly 200 pages, most of which are in example.com/category/product, however, the tool shows me that these are actually on 4-5 clicks away from ...
user avatar
  • 199
1 vote
1 answer
225 views

Screaming Frog hreflang missing confirmation links for rel="alternative"

Screaming frog tells me that I have missing confirmation links in my hreflang, but I cannot seem to identify the problem. Here is a screenshot of the code on our Korean page: Here is a screenshot of ...
user avatar
  • 13
2 votes
1 answer
63 views

Can you run screamingfrog on Google Cloud?

I've seen articles online on how to achieve this. They were written in 2016. Not too sure how relevant it is now? Has anyone tried?
user avatar
  • 21
2 votes
0 answers
177 views

Crawling a SalesForce site fully generated by JS with Screaming Frog only gets content on some pages

I'm trying to crawl a Salesforce-generated site. This site appears to be fully generated by Javascript (if you load the page with JS disabled only a white screen shows up). For instance, Salesforce's ...
user avatar
  • 357
1 vote
4 answers
454 views

Big discrepancy in number of known pages in Google Search Console and ScreamingFrog

In a technical SEO audit I used both Google Search Console and ScreamingFrog crawl to review the status errors, however there is big discrepancy in the number of pages that Google knows about, and the ...
user avatar
  • 199
1 vote
0 answers
94 views

Screaming Frog marking duplicate on noindex pages

Screaming Frog is detecting as duplicate all login pages of my site despite I have the directive robots-noindex. Do I have a problem on my site, or is just a false positive from Screaming Frog? It ...
user avatar
2 votes
1 answer
196 views

Screaming Frog removes custom request headers

We are working on an Angular SPA that uses token-based authentification (our client app gets a token from the server and saves it to localstorage). Then in each request to our API the client adds a ...
user avatar
3 votes
1 answer
4k views

Screaming Frog reports missing confirmation link when the hreflang x-default element is the same in all pages

I am confused about hreflang="x-default". I have a site in 2 languages. In each page I have a link to the page itself and a link to the same page in the other language. If I understand correctly, I ...
user avatar
  • 799
3 votes
1 answer
1k views

Checking site for meta refresh redirects

How do you check or crawl site (couple of urls) to explore existing meta refresh redirects? Screaming Frog doesn't handle them - it indicates a page with meta refresh as 200 (it is partly true), even ...
user avatar
  • 9,599
1 vote
1 answer
1k views

Image sitemap. Ok to use the ALT text as <image:title>?

Screaming Frog uses the alt text of an image when i create a sitemap of a crawled site and places it in the image sitemap as . Is this correct? Or should the actual image title be used as in the ...
user avatar
  • 1,117
0 votes
1 answer
221 views

Screaming Frog displaying deleted subdomain

In Screaming Frog, when I do a basic crawl of one of my website's in the 'Internal' tab it shows a sub-domain at the very top. This sub-domain was one I had previously created, but I removed it quite ...
user avatar
  • 113