Sparkle website very large

One of my Sparkle websites is large (850MB) - it has a lot of photos. Should I be worried? I am getting a lot of spinning beachballs and I need to ‘Force Quit’ more times than I would wish. However, I need to add that I have an ageing iMac27 (2012)!

850 MB should be Ok, even with an ageing Mac. However, if it is struggling it may be best to check the image sizes you’re placing in the project. You can drastically downsize the project file by optimising the images before adding them to Sàrkle. Download a copy of ImageOptim and run your images through that before adding to the project. This should keep the project file down quite a bit. Don’t worry about Sparkle optimising the images again - that shouldn’t happen because sparkle seems to have a smart image optimiser that stops it from over compressing images on output.

We have been looking into performance of large sites, and particularly on older Macs a lot of the times the system-triggered autosave causes Sparkle to spend a lot of time saving and compressing the file.

While we do wish to improve this, changing file formats is a slow process, definitely not in the initial Sparkle 3.

So as a troubleshoot (and temporary workaround) could you check if turning off autosave systemwide reduces the beachballs?

That can be done by going to the general system preferences and checking the “Ask to save changes…” checkbox.

I don’t generally recommend this, but if it makes things smoother it can be an option (and you need to remember to save yourself from time to time!).

This has long been an issue with my website of 2gb and 80 plus pages filled with images on El Capitan. Over time I have tried all kinds of workarounds. I cut the website down to four pages, that worked well. I began reducing the size of the images on the 2gb project but that soon got old and didn’t really make much of a difference. Duncan knows how much I bleated to him about the problem. Why I bleated can’t Sparkle do an Adobe Muse and upload changed pages only instead of the whole shebang?? But also I came to appreciate Sparkle’s style and super construction. So I recently threw up the 2gb version again and went with it. For a while I tried Duncan’s workaround with the system save again but that didn’t really work for me or Sparkle. The magic ingredient is process. Sparkle basically grabs the whole of my Macbook if I let it and knocks out the browsers, Preview, Indesign etc as well as invoking the spinning beachball but as long as you plan patiently around this it is as Duncan says manageable. I have got so good at this I hardly notice Sparkle is launched. When Sparkle decides it needs to autosave and take over the whole machine I just let it, and I go make a coffee or take a walk, but Sparkle seems mich better behaved lately. Over time I reckon Sparkle settles in and doesn’t seem to need to be such a grabber or else I am getting more patient. PRO TIP: I’ve found that if I quit Sparkle and let it do all its housekeeping in quitting, I can relaunch the project and publish pretty fast!!

Thanks for this … worth thinking about.

OK … but I cannot see an ‘Ask to save changes …’ checkbox. There is only an ‘Ask to keep changes when closing documents’, which is checked. This is in the ‘General’ section of Apple System Preferences.

Interesting. I will bear all this in mind.

Yeah the one. So you already have autosave turned off.

I strongly recommend you don’t turn off Autosave. I myself, and have seen people lose hours worth of work not remembering to Save. Autosave is not the problem. What you are experiencing is normal for a design workflow with Intel GPU’s and CPU’s. You are simply maxing out the RAM and CPU/GPU.

Design programs such as Sparkle, Apple’s Pro apps, and Adobe anything use all the RAM (memory), CPU, and GPU available. The age of a Mac only matters as far as the Intel CPU and GPU installed, which is thermal garbage. When you get the spinning beach ball, listen for your Mac’s fan.

To see for yourself, launch Activity Monitor and look at the usages by app. You can watch all your design apps fight for resources. Look at the top app, it will almost always be WIndowServer. This is the app that renders what you make your design app do and shows it on your screen.

The solution is to save money for the upcoming Apple Silicon Macs. Until then, my workaround is to do as much design work on my iPad Pro as possible.

OK … this all makes sense. I now need to consider whether it is worth updating to a new iMac27, but they are not cheap.

A temporary bandaid would be to max out your RAM if you haven’t yet done so.

I bigger concern though is your iMac won’t be able to run macOS Big Sur.

I could add to that… back everything up and then do a clean macOS install and then bring back all your backed up items.

Hunting around I found your iMac 2012 won’t be able to run Big Sur, nor Catalina if it is early 2012…

====================================
According to Apple, these Macs will run Big Sur:

  • MacBook 2015 and later
  • MacBook Air 2013 and later
  • MacBook Pro 2013 and later
  • iMac 2014 and later
  • iMac Pro 2017 and later
  • Mac Mini 2014 and later
  • Mac Pro 2013 and later

Macs that didn’t make Big Sur’s list but were on Catalina’s included the mid- and late-year 2012 MacBook Pro, mid-2012 MacBook Air, mid-2012 and late-2013 iMac, and late-2012 Mac Mini machines. The now-abandoned systems will be supported with security-only updates to the last-chance Catalina through the summer of 2022, however.

1 Like

I have decided to purchase a new iMac27 (16GB RAM, 1TB SSD) … and use the old one as an extension screen (since it is only worth £100 trade-in). This will hopefully sort things. I can’t wait for the new iMac Silicons … literally, I can’t wait … !

1 Like

If available in your locale, take a look at the “Refurbished and Clearance” section of Apple’s Website. You’ll save about 15% with full warranty. I buy from there whenever possible and have never had any problems.

Will do, thank you. I am in the UK and haven’t noticed a section like this before, but will look.

Hi Duncan I have late 2012 ima c3.4 GHz Quad-Core Intel Core i7, 32 GB 1600 MHz DDR3, NVIDIA GeForce GTX 675MX 1 GB running Catalina 10.15.7. It takes ages to start sparkle, Save files a nd to open font dialog box. with rainbow ball rotating. Often crashing too. I can not locate the "Ask to save changes box?. Where do I find it??? The website is ?large 1.65Gb. Any help would be welcome
Thanks

i have a problem that my MacBook Pro harddisk tells mee , it is full.
after some searching I found that the software makes autosave files of the compleet site, and not a change log file or something.
I save my site on an extra SSD card of 256gb.
wy does the site does not save there. ?
and the same problem is with uploading new content, the compleet site is uploaded and not only the changes, so it takes long to upload…


Hi @schubje,

while I appreciate that Sparkle is causing some frustration and headaches, your analysis does not reflect what is actually happening.

Specifically Sparkle relies on system functionality to save files, and it’s the system that (as an option) performs the autosave function.

From Sparkle’s point of view we only have a “save this file to this location” function, and the system calls it sometimes when you press CMD-S, sometimes when you drag the titlebar icon to an email (producing a snapshot of the current state instead of the previously saved version), and sometimes an autosaved version. Sparkle has no knowledge of where the saved data will be placed.

The autosave happens very frequently. Generally we observe it happening only when there is a change, or more specifically when Sparkle creates an undo history (so say moving something by 1 pixel or changing a setting), and one of three conditions is met:

  • several changes have been recorded
  • a few seconds have elapsed and the user is idle
  • the user switches away from Sparkle

So that’s potentially several autosaves per minute.

You already see that this doesn’t match the state of your Autosave Information folder, where there’s something from different days.

I don’t really know how those happened, but it’s the system mis-managing the autosaved files, not anything Sparkle does wrong.

I have observed from my own use that the system occasionally will forget open documents (not just in Sparkle), and leave them around in the Autosave Information folder, but I have never seen them with that hexadecimal name and .genstore.noindex extension, not sure what that is.

Someone in this Apple Community thread seems to have found a similar situation with Pages files:

https://discussions.apple.com/thread/5678737

Your suggestion of saving “change logs” isn’t really realistic as far as project file formats go, in the context of how people normally manage and exchange files.

You seem to infer that a since something is wrong with autosaving, something must be wrong with publishing.

So first, the two are completely unrelated.

Second, Sparkle does transfer all the files that need to be transferred to ensure integrity and cache consistency of the published site.

From the Sparkle documentation about publishing:

Sparkle’s goal when publishing is the performance of the published website. This has informed some choices that make the publishing process slower than what would be intuitive, but the end goal is a speedy experience for the site visitor.

For example:

  • during publishing Sparkle will use image compression algorithms that favor higher compression (and smaller file sizes), at the expense of compression time;
  • the main html file for each page, and the corresponding javascript and css files, are all re-generated every time the site is published, this is designed to work with server and browser caches.

In order to avoid re-compressing the images every time you publish, Sparkle associates a disk cache with each publish location, you can manually clear it from the Sparkle preferences window, but more importantly re-using the same publish location is what ensures publishing is fast and only transfers changed files.

Now your project file is extremely large (though we have customers with significantly larger files), depending on the contents you might have many files being published. We have found that some FTP servers choke on more than 10.000 files in a single folder, and the images folder Sparkle produces can sometimes have more than 10.000 files. In that case the FTP server is not telling Sparkle about all the files in the folder, so Sparkle assumes the files aren’t there and republishes them. This is a bit of an edge case but we are working on something that will address this.

Whatever the issue, we can troubleshoot publishing issues if you do the following:

  • close and reopen Sparkle (this clears the log you need to send)
  • publish once, wait for the completion (this establishes a baseline)
  • publish again, wait for completion (this shows what files were published as new)
  • use the “Send sparkle log via email” from the help menu

Send us that email.