Tuesday, August 6, 2013

ARC, pronounced "Argh"

When I built the previous (and still current) incarnation of Solitaire Till Dawn, I made the mistake of clinging to an older technology: I used Apple's Carbon API instead of their newer Cocoa framework. I had what I thought were good reasons to do so at the time, but in the long run it was a mistake (chronicled here, if you want the gory details). So this time, I am determined to do everything in the most modern way possible, doing my best to ensure the longevity of this version. In the past couple of weeks, this has caused me to take a bit of a detour to modernize an important part of Solitaire Till Dawn.

I'll explain what this is, but you'll need to be patient while I give you the background info you'll need to follow along.

One issue that every programmer has to face is memory management. Programs have to store and manipulate data, which takes up space in your computer's memory. Chunks of space get used, and as the program continues to run, may outlive their usefulness. Now, think of what your office would look like if you never threw out any piece of paper, ever, not even your scratched notes: it would soon become cluttered to the point where there was no room for more paper, nor for any work to get done. It's the same way in a computer program: you have to "throw out" the chunks of memory that are no longer useful. In the computer, this is more like erasing the paper so that it can be re-used; but it's still basically the same job of cleaning out the old stuff to make room for the new.

There are a number of ways to go about this. The original way that Apple recommended for their Cocoa framework would semi-automatically clean up some of the disused memory chunks, but required the programmer to keep track of which chunks were still in use, and to explicitly clean up some of the disused ones. The programmer had to say "okay, I'm done with this" for every chunk (except for some very temporary scratch chunks), or else the chunk might be kept forever.

A few years ago, Apple introduced a mechanism called "garbage collection" in which the Cocoa framework tried to do all the work of keeping track of which chunks were still in use, and which could be thrown out. This is a great system for a programmer, because you don't have to keep track yourself. You don't have to mark each chunk as disposable when you're done with it; you just forget about it and it gets cleaned up automagically. It's what I was using for Solitaire Till Dawn up until a couple of weeks ago.

But recently, Apple deprecated garbage collection: this means that, although it still works, Apple is warning us that it will stop working in some not-too-distant future release of Xcode and Cocoa. Apple is abandoning garbage collection because it is slow. Instead, they have come up with yet another way to manage memory, a faster and more efficient way, and this new way is now the recommended method for memory management in Cocoa. They now want all developers to convert their programs to the new way.

The new way is called ARC (which stands for "automatic reference counting", although you don't need to know that). Switching from older methods to ARC is not entirely straightforward: we developers have to make changes to our code in order to use ARC successfully. Apple thoughtfully provided a tool to aid us in converting our code to ARC, and my hat's off to them: the tool was incredibly useful. Unfortunately, it couldn't fix everything for me; in a number of places, it could only flag problems, and leave them to me to work out the fix.

There's a piece of functionality in Solitaire Till Dawn that has, for most of its history, used what is normally considered to be a bad programming practice. By "bad" I mean that the developer (me, in this case) needs to really know what he's doing in order to avoid certain pitfalls; and also that the code written this way may not be portable to new operating systems or CPUs. Well, I will claim that I know what I'm doing, and that code has been working fine in Solitaire Till Dawn for 15 years and more; but the day has come when indeed, it cannot be ported to a new architecture. It is not compatible with ARC.

The piece of functionality in question is the PCode engine, a hidden portion of Solitaire Till Dawn that is used in many of the more complex solitaires. It gives Solitaire Till Dawn a great deal of flexibility and it's the reason that Solitaire Till Dawn can contain such wildly different games as Klondike, TriPeaks, Thirteens, and Grandma's Game. I had to rewrite a big chunk of the PCode engine to make it compatible with ARC, and then I had to debug it.

To ensure that I'd done the job right, I had to play at least a couple of games of each of Solitaire Till Dawn's 100 solitaires. Some of the ones that are difficult to win required playing many more than just a couple of games, to be sure that they were working. And of course, some of them weren't working, and then I had bugs to fix. It was A Big Job, and it has taken me two or three weeks to get through it.

The good news is that it's done, and successful: I now have a fully up-to-date, ARC-compliant Solitaire Till Dawn, and it has no new bugs that I could find in over a week of searching. Yay!

The bad news is that it does have bugs. They weren't introduced by the ARC conversion (I know, because I kept a pre-ARC version for comparison), but of course they still must be fixed. In the end, the ARC conversion didn't cost me as much time as it might seem, because I knew I would have to test all 100 games again anyway. So now I've done that, and I have a list of the bugs I've got to fix. That's a good thing.

A side note: unrelated to (and prior to) the ARC conversion, some bugs cropped up in the behavior of the drawer that displays lists of games and players. I think they appeared when I upgraded to Mac OS X 10.8. I had to rewrite all that code, and that took several days. But that's done too, and was very successful.

My last post was about writing the built-in Help pages. That's still in progress; I put it on hiatus when Xcode started complaining that I was using a deprecated form of memory management. I plan to try to fix some or all of the bugs I've found next, after which I'll return to the Help pages.

As I've said before, it's a big, long job. I am spending as many hours on it as I can, every day, weekends included. We'll get there!


  1. Oh, thank goodness for this update. I LOVE Solitaire Till Dawn and I will wait as long as you need to get the new version. In the meantime I am playing an online game and going crazy because of its limitations compared to your program.
    Just by the way, I love the game itself, but I must tell you that many years ago I used to play Solitaire Till Dawn literally in the early hours of the morning, after I came home from the hospital where my mother was dying. I used to play and play until the sun rose, then I would grab some sleep and head back to the hospital for another day.

  2. So glad to hear that you're getting closer to completion! Your fans are waiting!

  3. Thanks for the update Rick, a really useful insight.

  4. There is light at the end of the tunnel! I can't wait until I finally get my favorite game back. And by the way, thanks so much for keeping us updated.

  5. Hang in there. We keep watching this site.

  6. Thank you for explaining the process you're involved in. Thank you for doing all this work for us. I've been a Solitaire Till Dawn player since 1994, I think, and I sorely miss it. It's wonderful to see that you're getting in sight of the release.

  7. Will the new STD also run on an iPad?
    Hope so.

  8. After upgrading to Lion, STD no longer worked, what a disappointment! I have down loaded a couple of other solitaire games. Nothing compares to Soltaire till Dawn. I'll be so happy, when STD is back!

  9. I can't tell you how much I appreciate all of your hard work, Rick. I will be worth the wait for all of us who are addicted to STD. I have never found a suitable substitute. Again, thanks!

  10. My husband threatened pulling out our old G5last night. Just so could play STD!
    Thank you for all your hard work. Looking forward to the new STD.