PapaJohn Productions

PapaJohn's Movie Maker 2 and Photo Story
Newsletter #69

Project Complexity and Memory Needs

The 'Windows Movie Maker cannot save the movie to the specified location' error message is all too common. It tells you to verify your source files are available, the saving location is there, and there's enough free disk space. You have all of those things, and it still can't save. We've learned through experience that the message isn't complete enough.
It's become routine to advise users that their project might be too complex, and to suggest breaking it down into parts, render the pieces, and then put them together into the final movie... and most of the time it works. One of my fee-based services is to take the complex project, do the project splitting, and return the sub-projects ready for rendering... I've never had that work-around fail because the issue was something else.
Why does breaking it down and rendering the parts work? Because there's a direct relationship between the complexity of a project and the amount of available memory. You can easily check the status of your memory, and watch it during the rendering to see how close it gets to the limit... even watch as the rendering crashes when memory usage reaches the limit, followed by the error message a couple seconds later.  
My Toshiba was in the shop for repairs... from August 1 to Sept 6... and I was working on my older less powerful Dell laptop. With only 144 MB RAM and less than a GB of available hard drive space, I took the opportunity to check project complexity against available memory. I'm still looking for the easy to use meter that tells me how close I am to the limit, before having to find out by doing the rendering. Lacking that, I'm trying to develop rules of thumb about project complexity. I'm not there yet, but this newsletter gets me a step closer.
... before getting into it, here are a couple notes...

My Toshiba laptop returned on Tuesday, with a new DVD/CD combo drive and TV Tuner. Both of them were replaced on the last trip just 3 months ago, and the TV Tuner had never worked since then. CompUSA told me they needed to reinstall the operating system from scratch if I wanted to see TV on the new tuner.
When it left this time, I told them to do what they needed for me to see the TV. It came back Tuesday with the operating system exactly where it was when I purchased the laptop in May 2003. It was working well and I could see TV, but I had more than a few software installations and updates to do.
The local CompUSA filed two requests to replace the laptop with another unit because of repeated issues... it wasn't approved. Maybe the 3rd request will be. It's 3 days since getting it back and my 'standard' issues started again tonight as I finished this newsletter.
.... on to the main topic

a Test Project
I started with an existing wmv file... the 'XML the Beginning' tutorial made recently by the Persian Poet Gal... a 2 minute and 10 second, 640x480, 602 Kbps, 7.6 MB file.
I could have started with anything... the goal was simply to make a project more and more complex, render it to a movie at each step, and see how much memory was needed.XML Beginning
Over and over, add more complexity after each successful render, assess the extra memory needed, and render again until I ran into the upper limit and got the error message.
the computer system
The Dell laptop has a 266 Mhz CPU, a 4 GB hard drive, and 144 MB of RAM. I bought it new in June 1998 with Windows 98.
In August 2004 I upgraded it to the Home edition of Windows XP.
I had no problems doing the upgrade and the laptop has been running the best it ever has... even though it's far short of the system considered minimal for XP and Movie Maker 2.

Starting the Project
Before opening Movie Maker, the Dell was using 116 MB of the 144 MB (80 percent of the installed RAM)... it wasn't going to take much complexity to use of the rest of the RAM and start digging into virtual memory...
Memory usage varies... I assessed things as I went using the Performance tab of the Windows Task Manager. Before starting a Movie Maker project, there are memory demands on the computer due to the operating system and other software running.
Opening Movie Maker 2.1 added 8 MB of memory to the total used... not much. The Windows Messenger service uses over 20 MB when I'm not even online. Sonic's MyDVD 4.5 uses 58 MB of memory when opened, before starting a DVD project.
Project StartAnother 24 MB of memory was needed to create the start of the project, to get it to the point shown at the left... the single clip of the starting wmv file was on the timeline and split into 5 sub-clips.
Total memory usage at this point was 148 MB, which was 4 MB over the installed RAM... the project was starting to use virtual memory. When it does that, there's a significant slowdown as using the hard drive is much slower than using RAM.
If you don't mind going that slow, you can keep going until the combined total of RAM and virtual memory reaches your established limit.

Virtual Memory
Virtual Memory - StartingThe virtual memory setting gave Windows XP as much as 432 MB of the hard drive space to use as if it was installed RAM... adding that to the 144 MB of RAM, the Dell could go as high as 576 MB of memory before it gets the message from Movie Maker about not being able to save the movie.
On this system, most or all of the memory available for rendering is from virtual memory, not installed RAM.
To change the setting, use Start > Control Panel > Performance and Maintenance > System > Advanced Tab > Performance Settings > Advanced > Virtual Memory > Change...
If we run out of memory, we can increase the virtual memory maximum size instead of purchasing more RAM... right?? Yes, we'll see.
Make a mental note of the Initial size of 432 MB... we'll mention it below.

Add Complexity and Watch the Effects on Memory Usage...
Anything added to a project adds complexity... and more complexity adds more demands on memory. How much?? We'll look.
Adding complexity happens routinely as you work on the project... keeping an eye on the effects of the complexity, and where you are relative to your memory limit by using your Task Manager...
Task Manager - Processes

Task Manager - Processes Tab
The Processes tab of the Task Manager helps you see what's using your memory, and lets you close down some processes that are not needed when you render a movie... that's one way to free up some additional memory if needed.
Movie Maker, with no project open, is using a bit over 12 MB of memory... the two lines starting with NAVA... are for Norton Antivirus, things I could close down to gain 6-1/2 MB of memory. I tend to leave them and other processes running.

Task Manager - Performance Tab
To track the effect of adding complexity, use the Performance tab of the Task Manager.
The Commit Charge (Memory Limit) is the key item to watch. Compare it to the other two numbers in the lower left block, the Total memory being used, and the Peak memory level reached since the computer was last started up.
This picture shows the laptop after a fresh reboot, with Movie Maker running but no project open.
Task Manager - Performance
It's telling me that the current memory demands from all running processes total 142 MB... that the current Limit is 356 MB... and that the highest or Peak demand for memory since the laptop last started up was 143 MB.
The 432 MB Initial Limit for Virtual Memory, noted above, shows up here in the 356 MB Limit (144 MB of RAM + 216 Initial Swap File size = 360 MB).
After checking the running processes and the memory needed for them, and setting up the Task Manager to watch things... it's time to add the complexity and note the effects. In doing it, realize that these numbers are precise, but things change so dynamically on the system that you can only use the numbers as approximations when trying to draw conclusions.

Rendering the Simple Project
To see how memory needs change with the choice of quality, I did some renderings without changing the project.
The memory used just before rendering was 148 MB. The memory needed to ramp up to the first couple percent progress points (I didn't spend the time to finish the renderings... the memory needed ramps up pretty quickly to the maximum level for the profile selected):
  • 1 MB limit - 170MB peak memory needed - very steady memory usage thru the rendering - it fell instantly to 155MB when rendering finished
  • 2 MB limit - went up to 181MB within a percent or two... and leveled... cancelled - fell to 160MB
  • 4 MB limit - went up to 194MB within 2 percent and leveled... cancelled - fell to 149MB
  • 8 MB limit - went up to 189MB within 2 percent and leveled... cancelled - fell to 152MB
  • 16 MB limit - went up to 251MB within 2 percent and leveled... cancelled - fell to 149MB
  • 32 MB limit (the max for the size setting was 33 MB) - went up to 268MB within 2 percent and leveled... cancelled - fell to 149MB
  • DV-AVI - went up to 200MB within 2 percent and leveled... cancelled - fell to 149MB
Here's a Quality versus Memory Neededchart to show these results... quality (file size) versus peak memory needed to render it.
The horizontal red line shows 46 MB was needed to render it to a DV-AVI file... the numbers in the vertical bars are the file sizes of the compressed WMV files, with the scale at the right showing the memory used to render them.
Maybe I made a mistake noting the memory needed for the 8 MB file... or maybe it really was an anomaly... it doesn't change the overall pattern so I didn't do a retest.
The bottom line is that, for a given project, rendering higher quality compressed WMV files needs more memory than rendering a DV-AVI file... and rendering a low quality compressed file needs less.
With the highest memory demands being for a high quality WMV file, I did all the remaining tests using the WMV option at the highest file size choice that Movie Maker offered... the maximum size allowed moved up as the project timeline got longer. As complexity increased, the size of the rendered movie went from the 1 MB limit above to one as large as 331 MB.

Adding Complexity
Here are the steps I used to make the project larger and more complex... and the effects on the memory needed. I looked at the changes in memory usage in two places. How much more was used when having the project open on the timeline, before the rendering? And then, how much more memory was needed to render the movie.
8 MB were used to open MM2, and another 24 to get the project started with its 5 clips. The rendering of the basic no-frills project to a 32 MB WMV file needed 120 MB. These are the baseline needs; I'll note the changes from there.
1 - added one random effect to each of the 5 clips... 14 MB more (60% increase) was needed for the project, but nothing extra needed to render it to a 32 MB movie.
This was the first clue that the addition of video effects has minimal effects on the memory needed for rendering.
2 - added a transition between each clip, 4 transitions total... with default durations... 24 MB more (60% increase) was needed for the project, and 17 more (15% increase) for the rendering to a 32 MB movie.
3 - maximized the transition durations by overlapping the clips as much as possible, reducing the total length of the movie. There was no change in the memory needed for the project, but 27 more was needed to render it, although the maximum file size choice had gone down from 32 MB to 20.
Steps 2 and 3 combined was the first clue that the memory needed to render transitions is more significant.
4 - added 5 more effects to the 1st, 3rd and 5th clips, to the maximum allowed of 6 per clip... the increased memory needed to work on the project was 9 MB and there was a decrease of 9 MB in the memory needed to render it to a 20 MB file.
Effects Added5 - added 5 more effects to the 2nd and 4th clips... only 1 more MB was used when working on the project, and 12 more to render it to the 20 MB movie.
Steps 4 and 5 combined reinforced what I saw in step 1 about video effects not needing more memory to render the movie.
6 - rebooted the computer... which cleared memory to the tune of reducing the total used by 34 MB.
This was the first of a few reboots during the testing, each one showing significant memory reductions. Something was happening to clear the air during reboots.
7 - added a single music wma file... the sample Beethoven clip. 4 more MB was needed for the project itself and 15 additional for the rendering to a 20 MB movie.
Adding music added more memory demands than adding effects, but less than transitions.
8 - added a single text overlay to the first 40 seconds... using the scroll perspective animation. 14 more MB was used to work on the project and 48 more to render it to a 20 MB movie.
Adding a text overlay required significantly more memory, especially to the rendering step.
This completed the basic project... it was now a 1 minute, 20 second timeline with 5 video clips, packed with special effects and transitions, and topped with background music and a long text overlay.
At this point I rebooted and jogged the total memory usage back down by 45 MB... after the memory usage was cleared with the reboot, the project open in MM2 was using only 12 MB more than before I started adding the clips to create the project. The rebooting had saved over 65% of the memory being used as the project was being built.
This drove home what I had seen of the memory savings from the first reboot in step 6.
The renderings weren't close to running into my total available memory limit. I'll change tactics to increase the complexity from here.

Doubling and Doubling Again... till it Crashes
9 - copied and pasted all the clips in the project... doubling the project duration to 2:40. It used 39 more MB to work on the timeline, but memory to render it to a 41 MB movie went down by 20 MB.
I was wondering if copying/pasting the same clips results in some efficiencies over having totally different clips... I don't know. Even if there are efficiencies, the complexity was going up due to overall project size increases.
At this point, each doubling of the project also doubled the file size limit for the saved movie... the same profile was being used each time.
10 - doubled again to 5:21. Memory used to view the project went up another 19 MB, and memory needed to render it to an 82 MB movie rose another 13 MB... very modest increases in memory needs.
11 - doubled again to 10:41. Only 4 more MB needed to view the timeline, but an increase of 59 MB in what was used to render it to a 165 MB movie.
12 - doubled again to 21:23. 27 more MB to view the timeline... and didn't know yet what was needed to render it to a 331 MB movie file.
The project had reached the complexity level needed to two-block the memory of the Dell laptop. It had
  • 16 repeating segments totaling:
    • 80 video clips
    • 6 haphazardly selected special video effects applied to each video clip - that's 480 total effects
    • 64 special transitions, most of them significantly longer durations than the default
    • 16 music clips
    • 16 title overlays 
Complex Project

The Can't Savememory to do the rendering reached the limit of 576 MB (144MB RAM + 432 MB of virtual memory = 576 MB)...
I got the familiar error message... in this case I was happy to see it so I could get on with the rest of the newsletter.
Memory usage ramps up pretty quickly in the rendering process, reaching close to peak levels within the first couple percent of progress... that saves a lot of wasted rendering time, as you can cancel the process and think about what to do.
If memory usage is over 95% of the Limit, you might still reach it at some point later in the rendering. If you have more slack than that, you should be able to finish. 
13 - I was increasing project complexity to reach this point... but instead of breaking it back down into parts to render separately, I wanted to go forward by raising the virtual memory limit.
I changed it to a maximum of 500 MB... 
Raised Virtual Memory
... and rebooted...
... and tried the render again.

The Final Render
With the new higher virtual memory setting, I started the final render.
It started at 01:34 am on Sept 5th... and took almost 30 hours to complete. Obviously I didn't watch it the whole time. Here are some of the data points I noted:
493 MB of usage at 0% with a projection of 981 more minutes...
500 MB and still at 0% with 1317 minutes left...
529 MB at 1% with 1709 minutes to go...
529 MB at 2% with 1950 more minutes projected
... 3% - 1991 minutes left (the projected time was still growing to this point)
570 MB at 23% (with a peak of 583 MB so far) with 1540 minutes to go
541 MB at 24% with 1505 minutes left
612 MB at 54% with 900 minutes to go
622 MB at 60% with 790 minutes left
605 MB at 64% with 708 minutes to go (with a peak so far of 632,616)
590 MB at 69% with 597 minutes to go
... 10:30 am on Sept 6th - the rendering had finished sometime overnight. The peak memory usage was 641,548, just a few MB less than the higher Limit... I pressed the Finish button and watched the movie. It played fine...

Cross-Check on Rendering to DV-AVI
My Toshiba laptop had returned, and I moved the project over to it... I wanted to confirm that rendering this complex project to a DV-AVI file used less memory than the high quality WMV option I'd been using. The chart early on in the newsletter says it should be less memory intensive.
It confirmed it. Peak memory usage during rendering to a DV-AVI file was 359 MB on the Toshiba, while rendering the WMV used 400, over 10% more (it probably was higher... I cancelled the rendering when it was at a few percent complete).

Conclusions and Closing
The rendering on the Dell finished successfully, but it took almost 30 hours to render a complex high quality 21-1/2 minute movie... Rendering is CPU intensive and the 244 Mhz laptop was running mostly at 100% CPU utilization throughout... an advantage of it being a lower powered computer is that it didn't overheat.
What are the lessons learned or reinforced by this week's testing?

Have a great week...