MacDevCenter    
 Published on MacDevCenter (http://www.macdevcenter.com/)
 See this if you're having trouble printing code examples


Java Programming on the Mac Transforming iCal Calendars with Java

by Daniel H. Steinberg
05/20/2003

Apple is using human readable text files to store the data from some of its most popular applications. At the same time, Java 1.4.1 on the Mac includes facilities for regular expressions. If you are a Perl programmer you might scoff that Java is for sissies. With exceptions and strong typing, Java makes you say "please" while Perl makes you say "sorry". Each has its place, but in this article we'll look at how well regex-centric work can be done in Java.

The "Open" iCal Format

You can store your iCal calendars on your iPod and check your appointments while enjoying music. But, as Terrie Miller wrote in Goodbye PDA, Hello iPod?, "to-do items from iCal don't synch." When you use iSynch to bring your iPod up to date, your iCal events are transferred but not your todos. Terrie's article was a reminder of why I haven't used my iPod as a PDA. Most of the items I need to keep track of each day are todos and not meetings. So, here's the plan: change all the todos to events, and then they'll synch.

How hard is it to change a todo into an event? Look for ics files in your home directory's Library/Calendars subdirectory. Open one of them in a text editor. If you're timid, make a copy and open the copy in your favorite text editor. Here's a sample calendar opened in BBEdit.

Screen shot.

You can pretty much figure out what's going on in the calendar. There are two events (my dog's birthday and a phone meeting with Derrick) and two todos (finish this article and lose some weight). Contrast that with the same information stored in a Microsoft Exchange file. This time inside of your home directory, open Documents/Microsoft User Data/Office X Identities/Main Identity/Database in a text editor. Here's a screen shot of the dog's birthday event.

Screen shot.

Tell me whether you can figure that out. Sometimes it takes a nudge or two to get going in the right direction. The second nudge was Matt Deatherage's April column on the back page of MacWorld magazine. The bar was set high for Matt as he was displacing Andy Ihnatko, one of the only Mac columnists worth reading. But in that first and in the subsequent articles Matt has gotten the wrong end of the stick in issues that have been dealt with long ago by people that actually understand them. The April article was taking Apple to task for pretending to have open formats where they really don't. The prime example used in the article was Keynote, Apple's presentation software.

I almost wrote this article about Keynote but the red herring was that the file format is XML with a published DTD. What makes Keynote accessible isn't so much that it is written in XML or that Apple has released its schema. The Omni group had written functionality into their outlining software that allowed you to transform an outline into a Keynote presentation before Apple published the schema. What makes Keynote and iCal accessible is that the files are stored as human readable plain text as Andy Hunt and Dave Thomas recommend in The Pragmatic Programmer. This is a book you should own and reread regularly.

Look at the first calendar image above. If the iCal format later changes or the product is no longer supported, we can still read the calendar file and easily construct software that can use that format as input. If Microsoft changes the Entourage format, it is unlikely that we can do much with the binary format of the second file. Sure we can pour over it and extract little pieces of text and try to reconstruct a schedule, but it won't be easy.

Mastering Regular Expressions

Related Reading

Mastering Regular Expressions
By Jeffrey E. F. Friedl

This article presents an example of how software can be written to work with an unfamiliar but semi-accessible format. Sure, if Apple changes the structure of iCal files, this application will break. The expectation is, however, that we will be able to fix it.

Here's the plan. Take an iCal file and read it from our disk into a String. Next, use the regular expression facilities now available on your Mac as part of Java 1.4 to change the todo items in our calendar to events. Third, take the altered calendar and save it under a different name so as not to overwrite the existing calendar.

This code is presented as an example. Do not use it on data for which you don't have a copy. It hasn't been widely tested. Consult a professionally trained computer scientist or a twelve year old child before attempting anything difficult on your own machine..

Reading your calendar from disk

If you are comfortable reading files from disk, you can skip this section.

The class ICalendarToString is fairly lightweight. Here's what it looks like to an outsider.

package ical;

public class ICalendarToString {
  public static String getCalendarAsString(String calendarName)
                       throws CalendarNotFoundException {}
}

It has a single static method, getCalendarAsString, that takes the name of the calendar as a String and returns the contents of the file in a String. You invoke it like this:

ICalendarToString.getCalendarAsString("example");

In the code listing below you can see the details. The body of the getICalendarAsString() method shows that three private methods are called. You can put everything into a single method, but that makes your code harder to understand. Note that when you read the body of this method, you get a high level view of what this class does. It locates the calendar file using the name it received, sets up a file reader that is used to read the contents of the file, and returns a representation of that calendar as a String. The method also throws a CalendarNotFoundException. It is important to use exceptions that help the developer know what has gone wrong in using your class.

package ical;

import java.io.FileNotFoundException;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;


public class ICalendarToString {
  private static final String CALENDAR_HOME = "Calendar/";
  private static File inputFile;
  private static FileReader calendarReader;

  public static String getCalendarAsString(String calendarName)
                          throws CalendarNotFoundException {
    findCalendarFile(calendarName);
    setUpFileReader();
    return representCalendarAsString();
  }

  private static void findCalendarFile(String calendarName)
                          throws CalendarNotFoundException {
    inputFile = new File(CALENDAR_HOME + calendarName + ".ics");
    if (!inputFile.exists())  throw new CalendarNotFoundException(
         "There is no file with the name " + inputFile.getAbsolutePath());
  }

  private static void setUpFileReader() throws CalendarNotFoundException {
    try {
      calendarReader = new FileReader(inputFile);
    } catch (FileNotFoundException e) {
      throw new CalendarNotFoundException(e.getMessage(), e.getCause());
    }
  }

  private static String representCalendarAsString()
                                 throws CalendarNotFoundException {
    try {
      char[] contentCharacterArray = new char[(int) inputFile.length()];
      calendarReader.read(contentCharacterArray);
      calendarReader.close();
      return new String(contentCharacterArray);
    } catch (IOException e) {
      throw new CalendarNotFoundException(e.getMessage(), e.getCause());
    }
  }
}

The awkward part of the code is found in the representCalendarAsString() method. You create a character array with enough slots to read in the contents of the file. You then use the calendarReader to read the file into the array, remembering to close the reader, and finally construct a String from this array.

The findCalendarFile method tries to find the calendar file in the directory that holds your calendars (you need to alter this to match your setup) and appends ".ics" to the end of the name entered. It may not be clear whether the user is to enter the calendar name as "example" or "example.ics".

If I were writing this application for distribution I would probably choose a strategy that took the complete file name as input and didn't add the suffix. In any case, you can see that the CalendarNotFoundException is thrown with a message that details the presumed location of the file. A developer that gets an exception, that "example.ics.ics" was not found, can quickly see that the calendar should be passed in as "example". In the other two methods, the exceptions are constructed to pass on the message and cause from the exception that triggered the condition. This uses another new facility from Java 1.4, chained exceptions. Here's the CalendarNotFoundException class.

package ical;

public class CalendarNotFoundException extends Exception {
  public CalendarNotFoundException(String description) {
    super(description);
  }

  public CalendarNotFoundException(String description, Throwable cause) {
    super(description, cause);
  }
}

Next stop is manipulating the calendar using regular expressions.

Much ToDo about Nothing

The idea is to use regular expressions to identify a todo item from our calendar and then replace the todo so that it can be recognized as an event. We could look at this all at once and use this as the search string.

BEGIN:VTODO(.*?)SUMMARY:(.*?)DTSTART(.*?)\n(.*?)END:VTODO

You could then use this as the replacement string.

BEGIN:VEVENT$1SUMMARY:TODO $2DTSTART;VALUE=DATE:20030520\nDURATION:P1D\n$4END:VEVENT

If you are comfortable with these type of expressions, then the code presented in this section will point you at how to accomplish this using the Java APIs. Our actual code may appear overly verbose to those already comfortable with regexen. The JavaDocs for these two classes contain a lot of information, but if you are going to end up doing a lot of regex work, you should pick up a copy of Jeffrey Friedl's book, Mastering Regular Expressions, 2nd Edition.

Consider this first pass at the code.

 public static String convert( String contents){
    Matcher todoMatcher = getTodoMatcher(contents);
    return todoMatcher.replaceAll(setTodoRegEx());
  }

  private static Matcher getTodoMatcher(String contents){
    Pattern todoPattern = Pattern.compile(getTodoRegEx(),Pattern.DOTALL);
    return todoPattern.matcher(contents);
  }
  private static String getTodoRegEx(){
    return "BEGIN:VTODO(.*?)END:VTODO";
  }

  private static String setTodoRegEx(){
    return "BEGIN:VEVENT$1END:VEVENT";
  }

The two classes used for regular expressions are in the java.util package. You create the patterns to match on using the Pattern class and do most of the matching and replacement work with the Matcher class. In the getTodoMatcher() method you create an instance of Matcher using the compile static method from the Pattern class. Then in the convert() method this matcher is used to replace all of the found matches with the replacement expression.

The expression in the getTodoRegEx() method will match a string that begins with BEGIN:VTODO and ends with END:VTODO. In between the dot will match almost any character and the star means match as many of those characters as you can. Ordinarily this would only apply to characters on the same line so this pattern would look for a string with that pattern that is contained in a single line. To enable the search to take place in multiple lines we have set the Pattern,DOTALL flag in the second parameter of the call to thecompile() method. This flag specifies that the dot should match any character including new line characters. Now the pattern can search across multiple lines.

That explains the dot and the star and the interpretation of the dot in the portion of the expression given by (.*?). If we didn't include the question mark we would be performing a greedy search. The effect would be that we would locate a string containing the first occurrence of BEGIN:VTODO and the last occurrence of END:VTODO. If you look back at our sample calendar, this would capture both todos in a single string. Worse, if there was an event between them it would capture that as well. To search for the first occurrence of the END:VTODO, we insert the question mark after the star. That leaves the parentheses. If we later want to refer to a bit of the found string we enclose what we want to refer to in parentheses. Now we can refer back to it. In this case the replacement string BEGIN:VEVENT$1END:VEVENT results in the following:

The first note is that you don't really get an event by switching the todos to events. We'll need to change the insides as well. We need to set the date of the newly created event and a duration. We'll set the date as today and the duration as a full day. For the most part you could use the search and replacement strings given at the top of this section although the date is hardcoded there. I find the following version easier to read and more modifiable and maintainable.

package ical;

import java.util.regex.Pattern;
import java.util.regex.Matcher;
import java.util.Calendar;

public class ICalendarTodoConverter {

  private static final String SAVED_PORTION = "(.*?)";

  public static String convert(String contents) {
    Matcher todoMatcher = getTodoMatcher(contents);
    return todoMatcher.replaceAll(setTodoRegEx());
  }

  private static Matcher getTodoMatcher(String contents) {
    Pattern todoPattern = Pattern.compile(getTodoRegEx(), Pattern.DOTALL);
    return todoPattern.matcher(contents);
  }

  private static String getTodoRegEx() {
    return "BEGIN:VTODO" + getTodoContentsRegEx() + "END:VTODO";
  }

  private static String setTodoRegEx() {
    return "BEGIN:VEVENT" + setTodoContentsRegEx() + "END:VEVENT";
  }

  private static String getTodoContentsRegEx() {
    return SAVED_PORTION // $1
        + getSummaryRegEx()
        + getDateRegEx()
        + SAVED_PORTION; //$4
  }

  private static String setTodoContentsRegEx() {
    return "$1"
        + setSummaryRegEx()
        + setDateRegEx()
        + "$4";
  }

  private static String getSummaryRegEx() {
    return "SUMMARY:"
        + SAVED_PORTION; // $2
  }

  private static String setSummaryRegEx() {
    return "SUMMARY:TODO "
        + "$2";
  }

  private static String getDateRegEx() {
    return "DTSTART"
        + SAVED_PORTION //$3
        + "\n";
  }

  private static String setDateRegEx() {
    return "DTSTART;VALUE=DATE:"
        + getCurrentDateAsString()
        + "\nDURATION:P1D\n";
  }

  private static String getCurrentDateAsString() {
    Calendar now = Calendar.getInstance();
    String rightNow = now.get(now.YEAR)
          + formatDateComponent(now.get(now.MONTH) + 1)
          + formatDateComponent(now.get(now.DAY_OF_MONTH));
    return rightNow;
  }

  private static String formatDateComponent(int i) {
    if (i < 10)
      return "0" + i;
    else
      return "" + i;
  }
}

The expression SAVED_PORTION is used for (.*?). Also you can follow the getxxx() methods to see how the calendar is pulled apart and compare to the setxxx() methods to see how it will be reassembled. For example you can see that we are inserting the words "TODO" as part of the summary. In get/setSummaryRegEx() methods you can see that the string SUMMARY: is going to be replaced by the string SUMMARY: TODO . You can use the Calendar class to format today's date but remember that the number corresponding to the month starts with 0 instead of the expected value of 1. Also, you will need to convert one digit months such as 5 for May to two digits 05 so that iCal can correctly parse the date.

Saving Back to Disk

Most of the work is done. We need to save the calendar back to disk, which amounts to reversing the process you followed when reading from disk. There is one more subtle point. Look back at the iCal picture from the first section. Notice this line.

X-WR-CALNAME;VALUE=TEXT:example

The calendar's name is stored as part of the text. Since we will be saving the calendar under a new name, we also need to change this value in the text. Once again we use reg ex to locate the string to be changed and supply the new value in the changeCalendarName() method. Here's the code for saving the calendar back to disk.

package ical;

import java.io.File;
import java.io.IOException;
import java.io.FileWriter;
import java.util.regex.Pattern;
import java.util.regex.Matcher;

public class ICalendarFromString {

  private static final String CALENDAR_HOME = "Calendar/";
  private static FileWriter fileWriter;

  public static void saveStringAsCalendarFile(String contents,
                                              String destinationCalendar)
                         throws CalendarNotSavedException {
    createCalendarFileWriter(destinationCalendar);
    contents = changeCalendarName(contents, destinationCalendar);
    writeCalendarToFile(contents);
  }

  private static void createCalendarFileWriter(String calendarName)
                            throws CalendarNotSavedException {
    File outputFile = new File(CALENDAR_HOME + calendarName + ".ics");
    try {
      fileWriter = new FileWriter(outputFile);
    } catch (IOException e) {
      throw new CalendarNotSavedException(e.getMessage(), e.getCause());
    }
  }

  private static String changeCalendarName(String contents, String destinationName) {
    Pattern calendarNamePattern = Pattern.compile("(.*?CALNAME.*?:)(.*?)\n");
    Matcher calendarNameMatcher = calendarNamePattern.matcher(contents);
    return calendarNameMatcher.replaceAll("$1" + destinationName + "\n");
  }

  private static void writeCalendarToFile(String contents) throws CalendarNotSavedException {
    try {
      fileWriter.write(contents);
      fileWriter.close();
    } catch (IOException e) {
      throw new CalendarNotSavedException(e.getMessage(), e.getCause());
    }

  }
}

Here we have used a CalendarNotSavedException that is exactly the same as a CalendarNotFoundException with the obvious name changes.

Putting It All Back

All you need is a entry point to the program. This Main just contains a main() that reads the example.ics calendar from disk, changes it, and stores it as ipod.ics.

package ical;

public class Main {
  public static void main(String[] args) {
    try {
      String contents = ICalendarToString.getCalendarAsString("example");
      contents = ICalendarTodoConverter.convert(contents);
      ICalendarFromString.saveStringAsCalendarFile(contents, "ipod");
    } catch (CalendarNotFoundException e) {
      System.out.println("Could not find Calendar example.ics: " + e.getMessage());
    } catch (CalendarNotSavedException e) {
      System.out.println("Could not save transformed calendar: " + e.getMessage());
    }
  }
}

The resulting file ipod.ics looks like this:

Screen shot.

It loads beautifully into iCal and synchs perfectly with my iPod. Now I can view my todos. There are many ways to extend this application. The main point is that with ICalendarToString and ICalendarFromString, you have the framework to convert a calendar into a form in which you can transform it any way you want. I should know better than to make promises about future articles, but another technique is to build a calendar object model and be able to manipulate events and todos without thinking at all about their persistence.

Daniel H. Steinberg is the editor for the new series of Mac Developer titles for the Pragmatic Programmers. He writes feature articles for Apple's ADC web site and is a regular contributor to Mac Devcenter. He has presented at Apple's Worldwide Developer Conference, MacWorld, MacHack and other Mac developer conferences.


Read more Java Programming on the Mac columns.

Return to the Mac DevCenter.


Copyright © 2009 O'Reilly Media, Inc.