Further to my previous post on Google Maps, I have figured out how to dynamically plot nodes in Drupal, with the aid of the GMap, Location, and Views modules (see a small example here - but be patient!). This took a bit of fiddling and I cound not find this collectively documented so here is what I did. This work is in relation to the development of the Scratchpads we are hosting at the NHM, but is applicable to anyone using Drupal 5.1+. Essentially I want users to be able to prepare datasets in spreadsheet software like Excel, upload them to their Scratchpad, and dynamically display their data in a Google Map Mashup - all at the click of a button.

In an ideal world we would prepare some minimal template Excel Files that users can download and populate with data, include corresponding import filters in the Scratchpads so users can upload these data, and some default "Views" anyone can instantly see the results of this work. For fields not included in the template, the CCK and Import modules are sufficiently easy to use that with some basic instructions, anybody could expand these templates to their bespoke needs, but by minimally using a basic template, we can collectively repurpose these data from all Scratchpad sites.

1. The dataset

Below is an example spreadsheet populated with some attributes typical of biological specimen data. This is real data showing the distribution of about 1000 seabird louse specimens. Ideally some of these fields (like taxon names) might tally with normalized data already held in a Scratchpad (this should be possible with future releases of CCK, and would need to be integrated with a classification in Drupal's taxonomy module) but for the moment this we will treat these data as freestanding. To format this for import, you need to save this as a Comma delimited CSV file in Excel.

GeoMashupTest.xls (Excel)

GeoMashupTest.csv (CSV)

2. Create a new content type

For this you will need to enable Drupal's Content Construction Kit (CCK) module, the Locations module, and the Import module. In short you need to create a new node type in CCK ("admin/content/types/add") that will take each record (row) in the spreadsheet, and define fields for each attribute (column) of the table. When creating the content type, you must associate the title field label with a unique value in your dataset (I chose the specimen record identifier Record_ID) but you can optionally ignore the body field label .I chose to leave this blank, as I wanted all my data to go into dedicated fields.

At this stage you also need to activate the Locations module for the node type you are creating (choose "1" from the pull down list for the maximum number of locations allowed for this type, and default number of location forms) and activate fields that correspond to the location data to hand. If you don't see the Locative information section, go to Access Controls (/admin/user/access) and check the submit latitude/longitude checkbox for your user type. Whilst there, make your permissions for the gmap_location, node_import, views and views_ui module are such that you can access these modules and site visitors with the right permissions can see the products of your work. Minimally your imported data file must include a decimal latitude and longitude for each record. You can also choose to include other location data (e.g., City, Country, etc) by ticking the appropriate check boxes, but take care as the system with reject data it cannot validate (see below). In my case I chose to import the Location Name and Country from my spreadsheet. Under the Workflow section you may also want to define a few other node properties. For example, all nodes much be published for the to be visible on a map, but I did not what these promoted to the sites front page.

Next define additional fields for the other attributes of your specimen records. I created the following fields and grouped them according to whether they were louse or host names:

You don't have to match field names, but if you do it helps with the next step.

3. Data import

With the import module activated, you should be able to instantly populate your site from the CSV version of your spreadsheet. Under Import Content (/admin/content/node_import) select the data file, choose Comma Separated Values (CSV) text file from the pull down menu, and select the node type (in my case Specimen Record). Then click the Mapping button and match the field names to the corresponding values in file you have just uploaded. Use the pull down menus and the example data to guide you through this. When complete, import the data and if successful, you will see the first 5 successfully imported records (minus the location data which is not displayed). Any records that did not import will also be displayed. If you try this with my data file, you will see some country names do not import as they don't validate correctly. I choose to ignore these, but I could have selected them for download and manually corrected the records.

4. View your data

Node Locations (WARNING - this link is very slow to load) in the menu will show all the nodes on your site that have location data including those you have just imported, but more likely you will want to create a custom map showing particular records sets (e.g. this one for the louse genus Harrisonialla). To do this you need to create a new View with the Views Module (/admin/build/views/add). Give this view a name (e.g. specimenRecords), place it on a page (as opposed to a block or a header etc), and choose GMap View from the View Type pulldown menu. Set the number of nodes to be displayed (type "0" to include all corresponding nodes), and then under Nodes add at least one field from the specimen data you have just imported. Content in these fields will appear when a user clicks on a marker in the map. Optionally, you can choose to create Filters for properties of the data, like a specimens taxon name, and you can even let users interact with the map by exposing these filters such that a user can limit markers to a particular selection.

Here is a screenshot of the node locations of the specimen records I uploaded:

Note that this view will take on the default map setting of your GMaps module so you'll need to be certain these are set correctly.

Caveats

Having played with this a little there are two major problems with this approach. The first is immediately apparent when you click on the real example of this map. (WARNING - this link is extremely slow to load) because there are nearly 1,000 nodes on this map and the table containing these data is not indexed. For that matter neither is the location table with the geolocation information. I can manually fix this but for real use we need an automatic solution for Scratchpad users. The other problem related to the GMap module. With many points it takes a while to render the marker overlays on top of the map. Clustering components of GMap module seem to help, but they don't always work (at least Google’s own Clustering solution doesn't). Clearly there is some more work to be done here.