Search This Blog

Friday, March 11, 2011

Designing for iPad, iPhone and iOS

As I have been working with the iPad I have learned a great deal about how this type of platform supports graphics.

The basic idea with the iPad (and iPhone and the other iOS devices) is that it uses special hardware to do a lot of things which would normally be done in software.  For me the model that works the best is Adobe Illustrator.

(As opposed to the "manual" model you see in the associated image.)

Basically you can think of what happens on the iPad display as if it where the pasteboard in Illustrator.  There are various objects which can be presented there - images, lines, boxes, paths, etc. - each with its own transparency, position, color, transformation and so on.

In Illustrator you think of each object (line, box, placed image) as a separate item - controllable with the little blue bounding and control points which appear when you select the it.  By dragging these control points or the object itself you can change its shape and position on the pasteboard.  (I don't know that there is a name for these objects in Illustrator other than an "Items.")

On the iPad its easiest to match this model with what is called a UIView.  A UIView is a iOS class that's used for manipulating the display.  I like to think of each UIView as a single Illustrator object because, just like Illustrator, each UIView has its own position, scale, rotation, transparency, and so on.

Just like Illustrator you can group these UIView objects together.  In the UIView world you do this by making a UIView a child of another UIView - sort of like Group and Ungroup in Illustrator.  You can also have nested UIView groups.  Moving the parent of the group moves all the children - just like moving a set of grouped objects.  Similarly you can scale and manipulate the UIView in other ways and the nesting and organization of the subviews is preserved.

Like Postscript RIPs UIViews are very fast doing some things and very slow at doing others.

The reason for this is that each UIView is associated with a display hardware context.  On the iPad there is both a computing processor and a graphics processor.  The graphics processor is used to move around bit planes (RGB color with transparency).  It can scale, rotate and manipulate the planes very efficiently in real time.

Each UIView is its own bit plane inside the graphic processor.  The graphics processor gives the computing processor a name for each object but the actual bits are stored inside the graphics processor.  So instead of the computational processor moving around the bits it simply tells the graphic processor the name of the object it wants to move (for example) and where to move it too.  The graphics processor does all the work.

This technique is used to create the various types of animations you see on the iPad display.  In the graphic arts world you can think of animations on the iPad display as if they were automated or scripted Illustrator options, e.g., select object A, move it to position B while rotating it from orientation C to D.  The UIView supports these automated operations directly where as in Illustrator you would have to use a scripting technology such as Apple Script.

On the other hand loading a UIView with a given image is very slow.  You can place PNG or JPGs into UIViews, but the cost in terms of performance is very high - so high in fact that you do not want to do it any more than necessary.  The reason for this is that on the iPad the computational processor is needed to load in the image, convert it to a form the graphics processor can use, and then load it into the graphics processor along with any other information needed (display position, transparency, etc.)

The Quartz display system on the iPad is basically a Postscript interpreter.  So in addition to images you can load what amount to (or use actual) PDF or Postscript-style descriptions of images.  These must be converted from PDF or Postscript representations to images which can then be loaded into the graphics processor.

For me, then, the best way to design iPad-type application interfaces is to use Illustrator and Photoshop.  I create items in Illustrator or Photoshop (which are treated as "Placed" in Illustrator) and organize them to meet the need of my UI.  Once the design is complete I simply map each element into a UIView.

No comments:

Post a Comment