Showing posts with label Core Data. Show all posts
Showing posts with label Core Data. Show all posts

Thursday, November 4, 2010

NSExpression

The relatively new NSExpression class is incredibly powerful, yet not really used very often. Part of that is that it's not very well documented. Although the API documentation for NSExpression is fairly well detailed, the listed "companion guide" (Introduction to Predicates Programming) has very little information about how to actually use NSExpression.

NSExpression deserves to be better documented, because it brings to predicate programming (including Core Data), a lot of features from the relational database world that people often complain are missing, like unioning, intersecting, and subtracting resultsets and performing aggregate operations without loading managed objects or faults into memory.

The aggregates functionality is especially important on iOS given the limited memory on most iOS devices. If you've got a large dataset, and you want to get a count of objects, or calculate an average or sum for one of the attributes, you really don't want to have to pull the entire dataset into memory. Even if they're just faults, they're going to eat up memory you don't need to use because the underlying SQLite persistent store can figure that stuff out without the object overhead.

I don't have time to do a full NSExpression tutorial, but I thought it at least worth posting a category on NSManagedObject that lets you take advantage of some of its more useful features.

With this category, to get a sum of the attribute bar on entity Foo, you would do this:
NSNumber *fooSum = [Foo aggregateOperation:@"sum:" onAttribute:@"bar" withPredicate:nil inManagedObjectContext:context];



This will calculate it for you using the database features, NOT by loading all the managed objects into memory. Much more memory and processor efficient than doing it manually.

Cheers. Category follows:

Header File:
@interface NSManagedObject(MCAggregate)
+(NSNumber *)aggregateOperation:(NSString *)function onAttribute:(NSString *)attributeName withPredicate:(NSPredicate *)predicate inManagedObjectContext:(NSManagedObjectContext *)context
@end



Implementation File:
+(NSNumber *)aggregateOperation:(NSString *)function onAttribute:(NSString *)attributeName withPredicate:(NSPredicate *)predicate inManagedObjectContext:(NSManagedObjectContext *)context
{
NSExpression *ex = [NSExpression expressionForFunction:function
arguments:[NSArray arrayWithObject:[NSExpression expressionForKeyPath:attributeName]]
]
;

NSExpressionDescription *ed = [[NSExpressionDescription alloc] init];
[ed setName:@"result"];
[ed setExpression:ex];
[ed setExpressionResultType:NSInteger64AttributeType];

NSArray *properties = [NSArray arrayWithObject:ed];
[ed release];

NSFetchRequest *request = [[NSFetchRequest alloc] init];
[request setPropertiesToFetch:properties];
[request setResultType:NSDictionaryResultType];

if (predicate != nil)
[request setPredicate:predicate];

NSEntityDescription *entity = [NSEntityDescription entityForName:[self className]
inManagedObjectContext:context
]
;
[request setEntity:entity];

NSArray *results = [context executeFetchRequest:request error:nil];
NSDictionary *resultsDictionary = [results objectAtIndex:0];
NSNumber *resultValue = [resultsDictionary objectForKey:@"result"];
return resultValue;

}

@end

Thursday, August 19, 2010

Core Data Starting Data

Yesterday, I tweeted asking for advice about the best way to provide a starter set of data in a Core Data-based application. The app I'm working on had started with just a small set of starter data, so the first time the app was run, I simply pulled in that starter data from a plist in the background and created managed objects based on the contents and all was good. The user never noticed it and the load was complete before they needed to do anything with the data.

Then the data set got quite a bit larger and that solution became too slow. So, I asked the Twitterverse for opinions about the best way to provide a large amount of starting data in a Core Data app. What I was hoping to find out was whether you could include a persistent store in your application bundle and use that instead of creating new persistent store the first time the app was launched.

The answer came back from lots and lots of people that you could, indeed, copy an existing persistent store from your app bundle. You could even create the persistent store using a Mac Cocoa Core Data application as long as it used the same xcdatamodel file as your iPhone app.

Before I go on, I want to thank all the people who responded with suggestions and advice. A special thanks to Dan Pasco from the excellent dev shop Black Pixel who gave very substantive assistance. With the help of the iOS dev community, it took me about 15 minutes to get this running in one of my current apps. Several people have asked for the details over Twitter. 140 characters isn't going to cut it for this, but here's what I did.

First, I created a new Mac / Cocoa project in Xcode. I used the regular Cocoa Application template, making sure to use Core Data. Several people also suggested that you could use a Document-Based Cocoa Application using Core DAta which would allow you to save the persistent store anywhere you wanted. I create the Xcode project in a subfolder of my main project folder and I added the data model file and all the managed object classes from my main project to the new project using project-relative references, making sure NOT to copy the files into the new project folder - I want to use the same files as my original project so any changes made in one are reflected in the other.

If my starting data was changing frequently, I'd probably make this new project a dependency of my main project and add a copy files build phase that would copy the persistent store into the main project's Resources folder, but my data isn't changing very often, so I'm just doing it manually. You definitely can automate the task within Xcode, and I heard from several people who have done so.

In the new Cocoa project, the first thing to do is modify the persistentStoreCoordinator method of the app delegate so it uses a persistent store with the same name as your iOS app. This is the line of code you need to modify:

NSURL *url = [NSURL fileURLWithPath: [applicationSupportDirectory stringByAppendingPathComponent: @"My_App_Name.sqlite"]];


Make sure you add the .sqlite extension to the filename. By default, the Cocoa Application template uses an XML datastore and no file extension. The filename you enter here is used exactly, so if you want a file extension, you have to specify it.

Since the Cocoa Application project defaults to an XML-based persistent store, you also need to change the Cocoa App's store type to SQLite. That's a few lines later. Look for this line:

if (![persistentStoreCoordinator addPersistentStoreWithType:NSXMLStoreType 
configuration:nil
URL:url
options:nil
error:&error
]
){


Change NSXMLStoreType to NSSQLiteStoreType.

Optionally, you can also change the applicationSupportDirectory method to return a different location if you want to make the persistent store easier to find. By default, it's going to go in
~/Library/Application Support/[Cocoa App Name]/
which can be a bit of a drag to navigate to.

Next, you need to do your data import. This code will inherently be application-specific and will depend on you data model and the data you need to import. Here's a simple pseudo-method for parsing a tab-delimited text file to give an idea what this might look like. This method creates an NSAutoreleasePool and a context so it can be launched in a thread if you desire. You can also call it directly - it won't hurt anything.


- (void)loadInitialData
{
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
NSManagedObjectContext *context = [[NSManagedObjectContext alloc] init];
[context setPersistentStoreCoordinator:self.persistentStoreCoordinator];


NSString *path = [[NSBundle mainBundle] pathForResource:@"foo" ofType:@"txt"];

char buffer[1000];
FILE* file = fopen([path UTF8String], "r");

while(fgets(buffer, 1000, file) != NULL)
{
NSString* string = [[NSString alloc] initWithUTF8String:buffer];
NSArray *parts = [string componentsSeparatedByString:@"\t"]

MyManagedObjectClass *oneObject = [self methodToCreateObjectFromArray:parts];
[string release];
}

NSLog(@"Done initial load");
fclose(file);
NSError *error;
if (![context save:&error])
NSLog(@"Error saving: %@", error);

[context release];
[pool drain];
}


You can add the delegate method applicationDidFinishLaunching: to your app delegate and put your code there. You don't even really need to worry about how long it takes - there's no watchdog process on the Mac that kills your app if it isn't responding to events after a period of time. If you prefer, you can have your data import functionality working in the background, though since the app does nothing else, there's no real benefit except the fact that it's the "right" way to code an application.

- (void)applicationDidFinishLaunching:(NSNotification *)aNotification
{
// Do import or launch threads to do import

// So, do this:
[self loadInitialData];

// or this:

[self performSelectorInBackground:@selector(loadInitialData) withObject:nil];

// But not both for the same method probably
}


Now, run your app. When it's done importing, you can just copy the persistent store file into your iOS app's Xcode project in the Resources group. When you build your app, this file will automatically get copied into your application's bundle. Now, you just need to modify the app delegate of your iOS application to use this persistent store instead of creating a new, empty persistent store the first time the app is run.

To do that, you simply check for the existence of the persistent store in your application's /Documents folder, and if it's not there, you copy it from the application bundle to the the /Documents folder before creating the persistent store. In the app delegate, the persistentStoreCoordinator method should look something like this:

- (NSPersistentStoreCoordinator *)persistentStoreCoordinator 
{
@synchronized (self)
{
if (persistentStoreCoordinator != nil)
return persistentStoreCoordinator;

NSString *defaultStorePath = [[NSBundle bundleForClass:[self class]] pathForResource:@"My_App_Name" ofType:@"sqlite"];
NSString *storePath = [[self applicationDocumentsDirectory] stringByAppendingPathComponent: @"My_App_Name.sqlite"];

NSError *error;
if (![[NSFileManager defaultManager] fileExistsAtPath:storePath])
{
if ([[NSFileManager defaultManager] copyItemAtPath:defaultStorePath toPath:storePath error:&error])
NSLog(@"Copied starting data to %@", storePath);
else
NSLog(@"Error copying default DB to %@ (%@)", storePath, error);
}


NSURL *storeURL = [NSURL fileURLWithPath:storePath];

persistentStoreCoordinator = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel:[self managedObjectModel]];

NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithBool:YES], NSMigratePersistentStoresAutomaticallyOption,
[NSNumber numberWithBool:YES], NSInferMappingModelAutomaticallyOption, nil
]
;

if (![persistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType configuration:nil URL:storeURL options:options error:&error])
{

NSLog(@"Unresolved error %@, %@", error, [error userInfo]);
abort();
}


return persistentStoreCoordinator;
}

}


Et voilĂ ! If you run the app for the first time on a device, or run it on the simulator after resetting content and settings, you should start with the data that was loaded in your Cocoa application.

Monday, August 2, 2010

Core Data Odds and Ends

I've been doing a fair bit of work with Core Data lately, and have picked up a few tricks to make my life easier, so I thought I'd share.

Mo' Power


Back in the EOF days, there was a great little piece of third-party software called EOGenerator that implemented something called the generation gap design pattern. Essentially, this little program would create your custom subclasses of EOGenericRecord (EOF's equivalent of NSManagedObject) for you. It wouldn't just create that, though, it would actually create two classes for each entity in your EOModel.

One of these two classes was yours to modify as you saw fit. It would never get overwritten even if you ran the EOGenerator script again, over and over. The other class contained all the generated code derived from your EOModel and was designed not to be touched by the programmer. You had your class, EOGenerator had its class, and you could both work without any fear of stepping on each other's toes. You could add attributes and entities to your data model and regenerate the machine-readable classes without any chance of losing your custom validation or methods. It was nice.

Wolf Rentzsch has re-implemented this idea for Core Data, and the result is called MOGenerator. If you spend any time with Core Data and aren't familiar with MOGenerator, go become familiar with it. It will make your life much, much, much better.

SQLite to Check Data


The default persistent store type for Core Data under iOS is a SQLite store, which means that Core Data keeps all of its data inside of a single SQLite database in your application's /Documents folder. Now, the format of the persistent store is undocumented and you should never, ever, ever, ever, never change data in the persistent store or make any assumptions about how Core Data stores the data. That's part of why we use Core Data: It's an abstraction layer that lets us not worry about the underlying storage details.

But, sometimes, it can be really helpful to know whether data is getting into the persistent store or not. I often work on my data model before my user interface, so being able to peek in the data store to make sure all is well is really helpful.

The first thing you might want to consider doing, if you plan to do that, is to create a SQLite config file. This is a file of commands that will execute whenever you launch SQLite. To do that, simply create a file called .sqliterc in your home directory. Here's what mine looks like:
.mode column
.header on
.table
In mine, the first line tells SQLite to print the column names when I issue a SELECT command. The second line tells SQLite to present the data in tabular form. The last line prints out a list of all the tables in the database before it shows me the SQL command prompt. That's handy given that the table names don't usually exactly match your entity or class names.

Once you have a config file in place, you can open up any persistent store using the sqlite3 command in Terminal.app, like so:
sqlite3 MyApplication.sqlite
You can't, of course, execute this directly against database files on your phone, but you can do it in the simulator as well as on files copied from your phone using Xcode's Organizer. If you're using the simulator, you can look for your SQLite databases at the following location:
/Users/[your account name]/Library/Application Support/iPhone Simulator/[SDK version #]/Applications/[application UUID]/Documents/[Application Name].sqlite


When I log into a database with my SQLite config file, this is what I see:
-- Loading resources from /Users/jeff/.sqliterc
ZHERO ZPOWER Z_METADATA Z_PRIMARYKEY
SQLite version 3.6.12
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite>
Notice that it has dumped the names of all my tables. It's pretty easy to figure out which table holds what data once they're right in front of you. The example above is the persistent store from the SuperDB application from More iPhone 3 Development. The Hero entity managed objects are stored in ZHERO and the Power entity managed objects are stored in ZPOWER. To look at the contents of a table, you just need to execute a SQL SELECT command. The simplest is just SELECT * FROM TABLENAME;, which prints the entire contents of the table to the console, like so:
sqlite> select * from ZPOWER;
Z_PK Z_ENT Z_OPT ZHERO ZSOURCE ZNAME
---------- ---------- ---------- ---------- ---------- ----------
1 2 2 1 Mutation Fire
sqlite>
The format of the data is pretty easy to figure out but, again, you really should look and not touch. It'd be very easy to accidentally destroy a persistent store. A seemingly inconsequential change could get the metadata out of whack and prevent your app from using the persistent store any more.

Better Descrption


I noticed a few days ago that Wil Shipley was complaining about the fact that many of Apple's delivered UIKit and Foundation objects have not overridden the -description method. This method is important because it's the one that is used in format strings, NSLog(), and when you type the po command in GDB. Instead of getting a meaningful description of many delivered objects, you instead get the class name and memory address of the instance, which is what NSObject's implementation provides, which isn't very much help while debugging.

Unfortunately, NSManagedObject is one of the many, many classes that doesn't have its own meaningful implementation of -description. That doesn't mean you can't provide one, though.

One thing you can do is to create an abstract subclass of NSManagedObject and put the -description method there. If you then subclass this abstract class, all of your managed objects will inherit the method from the abstract class. If I have other common functionality, I'll often do exactly that. But if I don't (and often you won't) I personally hate to clutter up my object hierarchy to add debugging tools. So I actually will cheat and do something you're not really supposed to do, which is override a method using a category. The reason it's bad to do this is because the Objective-C language does not specify which method should be used, which can lead to unpredictable behavior. But, when a class inherits a method from a superclass, I've found that overriding it in a category always works. I'm not suggesting you should do this in a shipping app, however. I wrap my method in pre-compiler definitions so that it doesn't get compiled into the release version of my application.

My version of -description assumes you're generating properties for your managed objects (I always do) and also assumes you're using custom subclasses (but you are, because you're using MOGenerator, right??). The method iterates over all the properties of the class and all its superclasses up to NSManagedObject, then prints them to the console. The categories I use look like this:
#if DEBUG
@implementation NSObject(MCManagedObjectDebug)
+ (NSMutableArray *)MCproperties
{
NSMutableArray *properties = nil;

if ([self superclass] != [NSManagedObject class])
properties = [[self superclass] MCproperties];
else
properties = [NSMutableArray array];


unsigned int propCount;
objc_property_t * propList = class_copyPropertyList([self class], &propCount);
int i;

for (i=0; i < propCount; i++)
{
objc_property_t oneProp = propList[i];
NSString *propName = [NSString stringWithUTF8String:property_getName(oneProp)];
if (![properties containsObject:propName])
[properties addObject:propName];
}

return properties;
}

@end


@implementation NSManagedObject(MCManagedObjectDebug)
- (NSString *)description
{
NSArray *properties = [[self class] MCproperties];
NSMutableString *ret = [NSMutableString stringWithFormat:@"%@:", [self className]];
NSDictionary *myAttributes = [[self entity] attributesByName];

for (NSString *oneProperty in properties)
{
NSAttributeDescription *oneAttribute = [myAttributes valueForKey:oneProperty];
if (oneAttribute != nil) // If not, it's a relationship or fetched property
{
id value = [self valueForKey:oneProperty];
[ret appendFormat:@"\n\t%@ = %@", oneProperty, value];
}


}

return ret;
}


@end
#endif
The +MCProperties method uses the Objective-C runtime to discover and iterate over properties in the class definition, including @dynamic properties that were created at runtime. Because the property runtime calls do not include properties inherited from parent classes, the method acts recursively, iterating up the object hierarchy until it gets to NSManagedObject. The actual -description method simply prints out each of the attributes and the value currently stored for that attribute for this object instance. Once you do this, if you use NSLog() or po, the result is much richer information about the objects getting printed to the console. This version ignores relationships and fetched properties, but it wouldn't be hard to add those in if you needed them.