Archive for the ‘Mac OS X’ Category

Retrospect w8

Monday, February 15th, 2010

There’s a rule of thumb for software development: make it, make it good, make it fast. For those unfamiliar with it, this means you should first build the core functionality of the software. Then, fix it’s bugs and make it as reliable as you can. Finally, optimize it to make it fast enough for your needs.

Retrospect 6 was a very good backup software, but its age was showing up. It was still fundamentally a Mac OS 9 app running on top of OS X Carbon API. The worst part about this was the need to launch the application in Finder, forcing you to have automatic login configured and remote-acess your backup machine via Remote Desktop or VNC, assuming it was running on a data center. Despite that, it was very reliable (I had two situations where Retrospect complained about corrupted data, and both situations were caused by faulty hardware). Also, it was not the fastest software I had seen, but it was good enough. Keep in mind Retrospect was designed when file systems had hundreds or a few thousand files on them, not a million or more like it’s normal today (I have about 1.5 million files on my laptop drive).

Facing the unavoidable, EMC decided to re-write Retrospect from the ground up as a modern OS X product, using Cocoa APIs and changing it’s architecture to a proper UNIX daemon (with a remote graphic console). Besides theoretically solving all the version 6 downfalls, they added some nice goodies, like AES 256 encryption, grooming, and some details. A nice one is that the disk backups are now stored in 100 MB files instead of a unique, giant file. This solves a lot of problems related to NAS systems.

Assuming you need backup software, you would think this were great news, right? Well, so did I. However, reality seems to be a lot worse than the perfect scenery I described above. EMC worked a lot of time on this version, and you would think they had made it, made it good and made it fast, right? Well, they decided to stop somewhere in the “Make it good” part. The problem is, rules of thumbs are nice, but common sense helps. If you launch a product that is simply too slow to be useful, people won’t use it.

Our backup machine at the university is an old G4, dual CPU (1 Ghz, I guess). Yes, it’s not exactly a screamer, but what the hell, we are talking about copying and storing files. It’s not exactly rocket science, and if the machine was up to the task when it was new, it should be up to the task now. We upgraded the RAM to a decent amount, of course.

So, last week I needed to recover the /etc directory of a colleague’s laptop to recover some apache and PHP settings that Snow Leopard installer happily overwrote. We’re talking about 3 MB of data, something that should be as simple as pressing a few buttons and get the data back. The laptop had about 750 thousand files on it, which, in my opinion, is not that many for today’s standards. So, why the hell took me about 3 hours to recover those files? Loading the catalog into the UI took almost 2 hours. Deselecting the whole file tree took almost 1 hour. The rest was the recovery process itself.

Ok, EMC. I know this stuff is optimized for Intel, and you do a lot of byte-order swapping on PowerPC. I know I’m using an old machine. But for god sake. What the hell are you doing to my CPU that needs 3 god damn hours to load a 750 000 files catalog into memory? And what’s the story with deselecting all the files? Are you telling me that, when I press the checkbox on the file root, you REALLY go trough the entire file tree and deselect each individual file? (In a rather inefficient way, because changing 750 000 booleans would take about… what? 1 millisecond on a 1 Ghz CPU?)

Well, I have at home a Dual 2 Ghz PowerPC machine that acts as backup storage, among other things. I have an AES 256 encrypted disk image (note that, at the university, I’m not using encryption, or else it would render things unbearable), served by AFP, acting as the time machine target for my laptop. When I installed Snow Leopard, I also had to recover some stuff from the /etc directory. Do you know how long it took? About 5 freaking seconds! And please, don’t tell me it’s because the G5 is faster!

Enough is enough, and 3 hours to recover a few files is ridiculous. I’ll study the possibility of migrating backups to Time Machine. Yes, space management is worse. Yes, client machines have to access the server, making the whole setup less secure than Retrospect where the server accesses the clients. But people won’t have to stop working for hours waiting for some files to be recovered.

Lesson for you, EMC: I’m not asking for you to make it fast before making it good, but at least make performance acceptable before releasing the product. Specially after taking so many years to build it.

Apple 2009 wish list

Friday, January 2nd, 2009

It’s a brand new year. So here’s my wish list for Apple:

  • Please fix the wireless driver that causes my Mac to crash about 10% of the times I turn Airport off.
  • Please fix the trackpad driver, or whatever is causing the trackpad to behave strongly erratic during about 30 seconds after waking the Mac up.
  • Please fix the damn copy/paste bug that makes the paste command paste the previously copied object and not the most recent one. This is specially irritating when you cut a piece of text, paste and you realize you are pasting something else, and that your supposedly cut piece of text is lost forever, unless you can undo and get it back.
  • Please fix the irritating bug that causes an iChat window to keep being the active one even after I click Safari, making its window go in front of iChat’s. That’s specially annoying when I type apple-W to close the Safari window, and the ichat one goes away.
  • Please provide replacement keyboards for people who has pre-unibody MacBook Pros that, you know, actually sense a keystroke every time the key goes all the way down, without the need to almost punch the key.
  • Please fix whatever is causing my father’s MacBook Pro to keep waking up and going back to sleep when the lid is closed and the charger on, despite I had already turned off every god damn thing that could wake it up, including the lid open event.
  • Speaking about the charger, please provide chargers where the charge light doesn’t go off for some unknown reason. It still works, but it doesn’t inspire a lot of confidence in it ans it’s safety.
  • Please provide granular updates to Mac OS X Server. Please please please pretty please.
  • Please care a little more about the entreprise and IT markets, namely your own web application technology (WebObjects, of course).

Thank you, guys! You must hate me but you’re nice people anyway. Sometimes.

Versions is out

Sunday, November 23rd, 2008

Versions is finally out! :) João Pavão, together with the Sofa team, released their new Subversion client. It’s a really powerful application made by people who deeply understand how a Mac application should be. In a world full of dubious software, it’s good to see that some people still care a lot about their code quality and the detail. I feel honored for having been one of the few who saw this application being born and getting mature enough to be released in the wild. Congratulations, João! :)

PGP Desktop 9.9 mini-review

Saturday, September 13th, 2008

Introduction

As I previously wrote in my blog, I was waiting for a whole disk encryption solution to be made available for Mac OS X. Some months ago, Checkpoint released what I believe was the first solution ever to support full disk encryption on the Mac, including the boot disk. Recently, as I noted before, PGP Corporation release PGP Desktop 9.9 for Mac OS X, supporting full boot-disk encryption for the first time on this platform. I opted for trying PGP, as they made a demo version available (that will work for 30 days) and it’s possible to buy a license online. Checkpoint, on the other hand, doesn’t seem to have a downloadable demo, and doesn’t sell the product online. This was enough for me to forget Checkpoint solution at all, specially having the PGP demo ready to be downloaded from their site and installed. So, PGP it is.

Why should you care?

Today, you easily buy a laptop with a large hard drive. That drive will quickly be full of important data. Many people consider “important data” to be private emails, trip photos, holiday movies, and such. But that’s just the tip of the iceberg. Letting anyone read your email or pick at your photos can be a problem, but it’s nothing compared to really important data. I’m talking about company reports. Source code. Data about your customers. Intelectual property. Financial data. Anything that keeps your business going, and that can put it in a very delicate position if it gets to the wrong hands.

Also, there’s an even more important fact you should take into account: some data you are carrying on your laptop is not yours, but other people’s data. Confidential emails with clients or business partners. Marketing and product information that should not be disclosed before a certain date. Governmental and military information, including private data from citizens (as an example, recently some events like this one happened in UK, where laptops with sensitive official information were stolen or lost). This means that you are no longer responsible just for your data security, but also for other people’s.

On top of this, you must think about the consequences of letting this information be revealed to the wrong persons. Important information about your clients may leak. Intelectual property that keeps your company ahead of the concurrency may become public, destroying your company advantage or, in the worst case, destroying the company itself. Governmental agencies may be placed in the hot seat for letting private information about the citizens be stolen and accessed.

All this together should be more than enough to make you worry about your computer’s data security and convincing you to do something about it.

Whole disk encryption

Computer security is a very wide subject, and there are a lot of things to consider. Network security, host security, etc. In this article, I’m covering whole disk encryption. So, what is whole disk encryption?

Whole disk encryption is a technique where all the drive contents are encrypted using a secret key (which can be a password, a key stored in a USB dongle, etc). When I say “all the drive contents”, I mean it. Even the operating system is encrypted. This means that you will only be able to access that drive’s contents if you have the key to access them, and this includes booting the computer from that drive. If you don’t have the key, you won’t be able to read the data whatever you try. Mounting it on other computers of physically installing it on a different computer won’t work. For anyone who doesn’t have the key, the drive will be as good as an empty one. All the contents will appear to be random garbage.

For those who have the right key, the main advantage of whole disk encryption is that it won’t affect the computer usage at all. The only thing you have to do is to type in the password right after powering up your machine. After you type the password, the OS will boot normally and the machine will work as if nothing special was happening. The secret is that PGP runs between the hardware and the OS itself, intercepting all the data input and output from and to the hard drive. All the applications, and even the OS itself won’t even realize that the hard drive is encrypted because the PGP layer will decrypt data requested by the applications on the fly. This is great, because it makes very unlikely that some application won’t work because of the disk encryption process. All the magic happens below the OS itself, as close to the hardware as it can be.

This will protect your data against one of the attack vectors that is most hard to defend yourself against: someone having physical access to (and some time alone with) your machine. This included the machine being stolen (which is very likely to happen at some point to laptop computers), someone entering your home or office and remove a hard drive from a computer, and even accessing data centers and stealing hard drives or entire servers (and if you think that doesn’t happen… think again, it’s more frequently that what most people believe).

I want to make clear that this will not protect your mac against other types of attacks. As I stated before, the OS and the applications will run in the same way they did before. So, if you have a virus or a trojan horse on your system, the virus or trojan will work. If you have a compromised network service, hackers will be able to get in using it. If you download an application that erases all your files, all the files will be erased. The whole disk encryption system has the only purpose to keep all the data on your hard drive protected when the system is not running. As long as you type in the password and boot the OS, all the OS-level security weaknesses that were there before will be there again. PGP Desktop has some more security features but I won’t cover them here.

What about Apple’s File Vault?

Apple provides you with some “transparent” data encryption features on Mac OS X, namely File Vault. File Vault will encrypt all the files in your home directory and store them on an encrypted disk image. You will always be able to turn the Mac on, but you must provide your account password on the login for that disk image to be accessed. As with PGP, data will be encrypted and decrypted on the fly. So, why not use it? There are many reasons why using File Vault is impractical:

  1. It’s not whole disk encryption, only home directory encryption. One may argue that all the important files are in the home directory, but that’s not entirely true. Many applications write temporary files to directories outside of your home directory, like /tmp. This files may contain sensitive information, and that information will be recorded unencrypted on your drive. Also, software like databases of other kind of servers store their data outside of user’s home directories, and that data will also be stored in clear.
  2. It conflicts with some applications, specially backup solutions. For any application executed by another user, including the OS itself, a user home directory will be a single, huge file, the encrypted disk image. The backup software will not be able to peek inside your home and only backup the files you changed since the last backup operation, so it will try to copy the entire file. Worse yet, if you change the file during the backup, you can corrupt the backup, making it hard or impossible to restore it if needed.
  3. For the same reason, remote services will not work because they won’t be able to decrypt your home directory. This is the case of a remote shell, for instance. If you ssh to a Mac with your home directory under the domain of File Vault, you won’t be able to access your files.
  4. It’s slow an unreliable. File Vault works by creating an encrypted virtual file system inside a file that grows and shrinks as needed that is itself stored in the real file system on your drive. There’s a huge load of things that can go wrong with this. This is corroborated by the fact that every time I tried to create disk images with many (hundreds of thousands) of files, the disk image inevitably corrupted and I could not access it’s content any more. Don’t forget that your entire home directory will be really a single huge file with some complex data and mechanisms that make it work. Now compare this with the simplicity of the PGP solution: just insert a layer between the OS and the drive, and don’t ever think about it again. It just works, it doesn’t need to care about files, folders, file systems, or anything else. It’s just raw data. OS asks it to write a sector on the drive, PGP layer encrypts the sector, and the sector goes to the drive. No complex processing, no complex data modeling, no complex code to fail. The PGP layer doesn’t even need to know what’s doing, it blindly encrypts and decrypts data on the fly. The OS will know what to do with that data.

For these reasons, I believe whole disk encryption is a much better solution than File Vault. I strongly believe Apple should provide this with their Macs right out of the box, but judging by the way the company handles security issues, I don’t believe that will happen any time soon.

PGP

What can you say when a product that is supposed to do what it does in the background and be totally transparent to the user actually works fine? Well, nothing. That’s precisely the point – providing security without being a pain to the user. So far, that’s my experience with PGP. I really have nothing much to say, except that it works.

I installed PGP, rebooted and typed in my demonstration registration key, valid for 30 days. Then, I read the manual, skipped all the “please verify your file system consistency before proceeding” warnings (what could go wrong?), set a password for my MacBook Pro drive, and fired up the encryption. You can use your mac normally while the initial encryption is done, as PGP is smart enough to know what disk sectors are already encrypted and which ones are not, allowing the system to work normally during the whole process. You will probably notice a very high loss of performance during the initial encryption process because the hard drive will be in really heavy usage (after all, PGP has to read and rewrite the entire disk surface).

After that, you won’t notice a thing. The only signs your mac will show you related to PGP are the small PGP icon on the menu bar, and, of course, the password window before the system boot. You won’t notice any performance degradation due to the real time encryption, at least I didn’t. Based on the UNIX “top” tool, it appears that PGP doesn’t use more than 2 or 3 percent of the CPU, which is negligible (remember that we are talking about 100% per CPU, which means that in a modern laptop with a Core 2 Duo processor, PGP is using 3%… of 200%).

You can create several “users” for your hard drive, with different passwords for each one. Please keep in mind this is only a way to avoid sharing the passwords. This is NOT a real accounting feature like in a normal UNIX system, where each user has different permissions and credentials to (supposedly) access only what he should. Here, any password will provide access to the entire drive contents. The normal access permissions will be granted by Mac OS X, of course, but PGP offers no data protection as soon as someone – whoever it is – types in a valid password.

Side notes

There are some important things to keep in mind when using PGP to encrypt your drive:

  • PGP asks for the password on system boot, but not on system awake. So, make sure to turn on the system awake password in the System Preferences, or you can allow a burglar to access your data if he stoles your Mac while it’s sleeping (and at least I always carry my laptop in sleep mode). If you want absolute security, turn off your mac before taking it with you.
  • Target mode (using your mac as a firewire drive) will work, but the drive contents will not be decrypted on the fly by the machine in target mode itself. You have to install PGP on the host machine so that it’s able to decrypt the contents of the target disk.
  • You have to be careful if you need to clone an encrypted drive. The most reliable way to do it is to decrypt it before cloning. If you want to clone an encrypted drive, check this thread in the PGP Forum for more details.
  • I haven’t tested this, but from what I recall from the manual, you may install PGP on a machine and use it unlicensed to read the contents of an encrypted drive (assuming you know the password, of course).
  • BootCamp won’t work. If you need Windows, you have to run Parallels, VMWare or any other virtual machine software. Those will work fine, providing that the Windows disk image is a file on the OS X file system, and not a dedicated Windows-formatted partition.

PGP Desktop 9.9 released

Monday, August 25th, 2008

PGP finally released the 9.9 version of it’s PGP Desktop product I had mentioned before, featuring full boot-disk encryption for OS X (Intel only). The demo version is not yet available, but I can hardly wait to try it. I hope they handle correctly booting in target mode and verbose or single user mode.

PGP Whole Disk Encryption for Mac OS X

Wednesday, June 25th, 2008

PGP is about to release a new version of it’s PGP Whole Disk Encryption product. This version will fully support the Mac, including the boot disk. I’m looking for a full disk encryption system for a long time now, as it’s a really nice solution in a world where the number of stolen laptops keeps increasing. This won’t help at all recovering a stolen Mac, but at least gives you peace of mind about the data in your stolen computer drive. The drive contents will appear as random data for anyone who doesn’t own the PGP password, so you know that, despite having to buy another laptop, your data won’t be seen by anyone else. As we carry more and more vital data inside our laptops (not just the data itself, but also passwords and ssh keys that allow access to servers with even more important data), protecting all that stuff in case of theft of loss is becoming more important every day.

I hope they release a demo version, as I would like to know how the system reacts when awaking the machine from sleep and using target mode. Until then, you can register yourself at the PGP site to be notified on the release date.

Back from USA

Sunday, June 15th, 2008

Well, I’m back from another WOWODC and WWDC. I’m still really tired, but some quick notes:

  • As David LeBer already mentioned, Pascal did an amazing job organizing WOWODC all by himself. Great room (a bit cold on the first day ;) ), large windows and sunlight on the halls. Food (well… not that good, but after all, it’s USA!) and caffeine provided frequently. Very very nice. Suggestion for the next year: a bigger (and brighter) screen, and eventually plasma screens among the room to make it easy for people in the back to read the code.
  • I learned a lot about WO frameworks out there (like Wonder, Houdah and specially LEWOStuff that I did not know before). I met for the first time some very talented people, and of course, all the folks from the previous conferences. It’s great to be able to have technical discussions and know different views on the same problems from all those skilled and experienced people out there, face to face.
  • WWDC had some interesting news on many stuff. As you know, I cannot talk about the stuff under NDA, so I shall only say that some interesting stuff is being done on the WO side. Also, as you all know by now, the iPhone is now 3G, includes a GPS, the price was slashed, and will be available in many countries of the world. I just hope the service providers slash the data roaming prices, because that makes the iPhone useless when you go to foreign countries. Finally, Snow Leopard was announced, and, as already expected, the focus is not on new features, but on a big cleanup of the OS infrastructure. Not only this are great news for us, developers, but also shows some courage from Apple and a lot of respect for their users. They want to focus the next year on improving the quality of their OS, rather than packing it up with some new features just to win the race against the competition.
  • As a side note, the MacTech people was giving away some magazines for free to the people who were standing in line during the morning. I took the time to read most of it during my flight, and I really liked it. I was a MacTech subscriber in the past, but I cancelled it because, during my graduation, I didn’t have time to read it (it’s good to graduate on a place where you actually don’t have time to learn, isn’t it?). Maybe I’ll subscribe it again now.
  • Not related to the conferences themselves, we went to visit the bay area surroundings on Friday afternoon. We did the classic trip to the Apple and Google campuses, because we are all geeks, but we also went to the Stanford and Berkeley campuses. The Standford campus totally blow me away. You have to see it to believe it. From now on, I’ll laugh, really laugh, every time I hear a faculty from my university stating that we actually have a campus. The Berkeley campus did not impress me much. It’s more urban style, more crowded and dense. I prefer the Stanford way, with space, a huge amount of space, tons and tons of space, really. Almost made me want to return to the univ! :)

Accessing Mac virtual hosts from a Parallels VM

Sunday, March 30th, 2008

I finally moved to an Intel machine. Despite the dramatic speed improvement in everything Java-related, namely Eclipse, there’s another big advantage: being able to run IE on Windows using a virtual machine. Unfortunately, that’s something every web developer must do to ensure his or her application will work on the most used (and crappy) browser on earth.

I installed Parallels and created two virtual machines, one for IE 6 and another one for IE 7. This way I’m sure there are no weird problems between those two versions (having more than an IE version on Windows can only be accomplished by hacks, and hacks are bad). Also I can install Visual Web Developer Express Edition on each of the VMs, and use either IE 6 or 7 to debug.

My apps run inside virtual hosts on Mac OS X apache, under a fake DNS name. On Mac OS X it’s easy to add the DNS entry to the /etc/hosts file, under the 127.0.0.1 entry. This way, your DNS name will always point to your mac, and you’ll be able to reach your virtual host.

I wanted to do the same from inside Windows running on Parallels. An easy way would be to edit the Windows hosts file, adding the Mac OS X public IP to the file. But that will only work if the OS X IP doesn’t change. My Intel mac is an MBP, and I change the network I use often, so I needed a little more flexibility. So, this is the way I found to do this:

  1. Configure your VM to use Shared Networking. This wall, Parallels extensions installed on your Mac will create a NAT network where your virtual machine will be hooked into.

    Paralleles Configuration Screen
  2. Open Mac System Preferences, and look for the “Parallels NAT” network port. This is an interesting one, because it allows the Mac itself to be connected to the virtual NAT network, using an IP on the NAT subnet. Write down that IP: this will be the IP you’ll use to access the Mac virtual hosts from within the virtual machines.

    System Preferences
  3. Finally, edit Windows hosts file. This file is located on \WINDOWS\system32\drivers\etc\hosts. Add a line with the IP (in my case, 10.211.55.2) and the name of the virtual host, just like you do on the Mac.

    Windows hosts file

That’s it. Now you can access your Mac virtual hosts from Windows, whatever the Mac IP is. Ick, what’s a Windows screenshot doing in my blog!?

drawImage performance on Leopard

Wednesday, February 27th, 2008

A not very fast but handy way to downscale images in a WebObjects application is using Java 2D APIs, with code like this:


BufferedImage reducedImage = new BufferedImage(newX, newY, BufferedImage.TYPE_INT_RGB);
Graphics2D g = reducedImage.createGraphics();
g.setRenderingHint(RenderingHints.KEY_INTERPOLATION, RenderingHints.VALUE_INTERPOLATION_BICUBIC);
g.setRenderingHint(RenderingHints.KEY_ANTIALIASING, RenderingHints.VALUE_ANTIALIAS_ON);
g.setRenderingHint(RenderingHints.KEY_RENDERING, RenderingHints.VALUE_RENDER_SPEED);

g.drawImage(originalImage, 0, 0, newX, newY, 0, 0, originalImage.getWidth(), originalImage.getHeight(), null);
g.dispose();

Although not blazing fast, this is enough for many applications. I could reduce a 7 Mega-Pixels image to something like 250 pixels wide in about one second, or less, in my PowerBook G4. But this was in Tiger.

In Leopard, as some of you may have noticed (and if you have applications deployed on Leopard Server, be aware) this is incredibly slow. When I say slow, I say five minutes, or even more, with the CPU being used at 100% during that time.

There are two reasons that lead to this. The first (which is not a problem in itself, but it’s a cause of the problem): Apple switched from Quartz to Sun2D graphics engine as the default one for Java applications on Leopard. So, all your image manipulation is being done using the Sun pipeline now. This would not be a problem, except for the second reason: the Apple JVM implementation has a bug that is slowing Sun’s pipeline drawImage method to a crawl. Actually, that was not the real reason. I testes this on FreeBSD (using Diablo JDK) and the speed was similar to Leopard’s. Sun2D is REALLY slow, to the point of being useless. I’m now using ImageMagick.

The only solution for now is forcing the application to use Quartz engine. You can do that using the command line option -Dapple.awt.graphics.UseQuartz=true. And, of course, file a bug on this!

Leopard tech talk, Lisbon

Tuesday, December 4th, 2007

Yesterday I spent all day in the first ever Apple developer event in Portugal. Apple carried out a Leopard Tech Talk in Lisbon, where portuguese developers could learn about some of the new stuff in Leopard, including 64 bit programming and Core Animation. The speakers were splendid, and with great technical knowledge about what they were talking about. It’s always great to watch a technical presentation made by real coders, and not by the full-of-bullshit marketing people.

Some presentations were very superficial, but the most interesting ones went as deep as some of the WWDC sessions I attended. The event was actually a micro-WWDC, and even included a nice buffet with plenty of food for lunch and coffe-breaks, all for free. As I spend almost all the WWDC week on the IT track, it was cool to learn about the desktop stuff Apple is working on.

The room was packed, and people were motivated and participative. The Apple guys really liked that, as they say those are the main factors they use to evaluate how successful and event it, specially when going to a new country for the first time. I know I’ll be there next time!