Geoloader
Geoloader automates the process of loading GeoTIFFs and Shapfiles into Geoserver, Geonetwork, and Solr (using the OpenGeoPortal schema), the services that power the geospatial search interface at the University of Virginia Library.
Quick Examples
geoloader solr
Load files to Solr.
# Load files matched by wildcard:
geoloader solr load /path/to/files/*
# Load an individual Geotiff:
geoloader solr load /path/to/geotiff.tif
# Load an individual Shapefile:
geoloader solr load /path/to/shapefile.sh
# Load files to a custom workspace:
geoloader solr load /path/to/files/* --workspace aerials
# Merge markdown metadata into the Solr documents:
geoloader solr load /path/to/files/* --description /path/to/markdown
# Push the jobs onto a Resque queue:
geoloader solr load /path/to/files/* --queue
# Clear all documents in a workspace:
geoloader solr clear workspace
geoloader geoserver
Load files to Geoserver.
# Load files matched by wildcard:
geoloader geoserver load /path/to/files/*.tif
# Load an individual Geotiff:
geoloader geoserver load /path/to/geotiff.tif
# Load an individual Shapefile:
geoloader geoserver load /path/to/shapefile.shp
# Load files to a custom workspace:
geoloader geoserver load /path/to/files/*.shp --workspace aerials
# Push the jobs onto a Resque queue:
geoloader geoserver load /path/to/files/* --queue
# Clear all documents in a workspace:
geoloader geoserver clear workspace
geoloader geonetwork
Load files to Geonetwork.
# Load files matched by wildcard:
geoloader geonetwork load /path/to/files/*
# Load an individual Geotiff:
geoloader geonetwork load /path/to/geotiff.tif
# Load an individual Shapefile:
geoloader geonetwork load /path/to/shapefile.sh
# Load files to a custom workspace:
geoloader geonetwork load /path/to/files/* --workspace aerials
# Merge markdown metadata into the ISO19139 records:
geoloader geonetwork load /path/to/files/* --description /path/to/markdown
# Push the jobs onto a Resque queue:
geoloader geonetwork load /path/to/files/* --queue
# Clear all documents in a workspace:
geoloader geonetwork clear workspace
Or, from ruby
# Load a Geotiff to Geoserver:
Geoloader::GeotiffGeoserverLoader.new("/path/to/file", "workspace", "/path/to/desc.md"}).load
# Load a Geotiff to Solr:
Geoloader::GeotiffSolrLoader.new("/path/to/file", "workspace", "/path/to/desc.md"}).load
# Load a Shapefile to Geoserver:
Geoloader::ShapefileGeotiffLoader.new("/path/to/file", "workspace", "/path/to/desc.md"}).load
# Load a Shapefile to Solr:
Geoloader::ShapefileSolrLoader.new("/path/to/file", "workspace", "/path/to/desc.md").load
# Load a Geotiff or Shapefile to Geonetwork:
Geoloader::GeonetworkLoader.new("/path/to/file", "workspace", "/path/to/desc.md").load
# Or use any of the loader classes as a Resque job:
Resque.enqueue(Geoloader::GeotiffSolrLoader, "/path/to/file", "workspace", "/path/to/desc.md")
Installation and Configuration
To get started, clone the repo and install the gem:
rake install
Then, you'll need to point Geoloader at running instances of Geoserver / Solr / Geonetwork (or any combination thereof). By default, Geoloader starts by applying the configuration settings in the top-level config.yaml
file:
workspaces:
production: geoloader
testing: geoloader_test
solr:
url: http://localhost:8080/solr/geoloader
geoserver:
url: http://localhost:8080/geoserver
username: admin
password: geoserver
srs: EPSG:3857
geonetwork:
url: http://localhost:8080/geonetwork/srv/en
username: admin
password: admin
group: geoloader
Depending on your needs, you can override some or all of these settings. For example, you'll almost always need to set custom credentials for Geoserver.
Ruby
If you're using Geoloader programmatically in code, pass any hash-like object to the configure
method on the Geoloader
module to set configuration options directly:
require "geoloader"
Geoloader.configure({
:geoserver => {
:username => "gs_username",
:password => "gs_password"
}
})
Or, put any combination of custom settings in a separate YAML file:
geoserver:
username: gs_username
password: gs_password
Which are automatically applied to the configuration when the gem is loaded.
Descriptive Metadata
Geoloader also makes it possible to provide text metadata about assets in Markdown files, which are parsed and merged into the records that are pushed out to the services. These files have three parts:
-
A YAML "front matter" section, delimited by
---
dividers. This makes it possible to provide arbitrary key-value metadata, which can be merged into records created by the concrete asset modules. -
A leading
<h1>
element, represented in Markdown as a line that starts with a single#
- eg,# Testing Title
. This value is used as the generic title for the asset(s). -
The rest of the Markdown document, which is used as an "abstract" or "description."
For example, here's what a description file for an upload to Geonetwork might look like:
---
categories:
- category1
- category2
keywords:
- keyword1
- keyword2
---
# Testing Title
A testing abstract! More content here.
CLI Application
If you're using Geoloader as a command-line tool, provide custom settings in ~/.geoloader.yaml
, and the values will automatically be merged into the default configuration at runtime:
# ~/.geoloader.yaml
geoserver:
username: gs_username
password: gs_password