Update 1.44 – Screaming Frog, User Agents, & Big Files
As well as a few new features and bug fixes, this update includes some big improvements to the way URLs are downloaded and the HTML is processed. It’s the most stable release yet.
Automatic Connection Optimisation
So we all know the “tortoise and the hare” story. Well it turns out that running a profiler flat out doesn’t make it any quicker. It’s actually the opposite in all the tests we’ve done.
By default, URL Profiler now optimises its connections based on your computer’s CPU and available cores. This will make it quicker, more stable and return reliable results.
It’s not recommended, but you can turn this feature off.
Screaming Frog SEO Spider Import and Combiner
To make it easier for you to content audits, we’ve added a new function that imports from Screaming Frog SEO Spider export. When this function is used, URL Profiler’s data is appended to the end of each matching URL. Basically merging the data together to give you a combined file to work from.
User Agent Selector
You can now select what User Agent is when requesting URLs. The selection is made within the settings panel.
Maximum File Size to Download
Within the connections settings you can set the maximum file size of a URL to download. If set URL Profiler will stop downloading the file when your specified size is meet. There’s a lot of rubbish out there on the interweb, it’s not uncommon for a spammed URL to exceeding 4MB with 10,000+ comments. According to Google an average HTML file, just the code itself is 320KB. We recommend setting your maximum download size to 1024KB.
Updates
Completely 64 bit
The entire system has been ported over run on 64 bit machines, and gracefully downgrades for those still on 32 bit. Without getting into the specifics this is a good thing!
Content Hash
The hash is computed based on the content text first and if that fails it reverts back the body text. This is a more accurate representation of the content, help you to identify duplicate URLs.
Google Analytics Connection
Improved the layout and messaging within Google Analytics connection window, making it easier to connect to your accounts.
Removed Proxy Nagging
Based on some feedback we’ve improved the messaging around proxies – no more nagging!
Frozen Process Interrupt
The system will now gracefully abort a process if URL Profiler thinks it is frozen and is hanging on the final few URLs.
Bug Fixes
- Reduced the HTTP redirect capturing to fix redirect loops
- OS X Fixed the intermittent import from file error
- OS X Sped up the import and paste function
- Pinterest incorrect figures at domain level. Didn’t like trailing slashes!
Downloads
If you’ve not tried URL Profiler yet, you can start a free 14 day trial here:
The trial is full featured, and you don’t need to give us any payment details to get started.
Existing customers or existing trial users can grab the new update from here:
What are you waiting for?