Warclight is no longer under active development. This repo has been set to public archive.

If you're interested in web archive collections and Apache Solr, check out SolrWayback. It uses the same underlying library as Warclight to index web archives into Solr, and offers many more features than Warclight.

<3 AU Team

A Project Blacklight based Rails engine that supports the discovery of web archives held in the WARC and ARC formats. It allows faceted full-text search, record view, and other advanced discovery options.

The following article provides a nice overview:

Ruest, Nick, Milligan, Ian, and Lin, Jimmy. Warclight: A Rails Engine for Web Archive Discovery. Proceedings of the 2019 IEEE/ACM Joint Conference on Digital Libraries (JCDL 2019), June 2019, Urbana-Champaign, Illinois.

• Ruby 2.6 < 3.0
• Bundler < 2.2
• Rails 5.1 < 6.2
• Solr < 9

Add this line to your application's Gemfile:

gem 'warclight'

And then execute:

$ bundle

Or install it yourself as:

$ gem install warclight

For further details, see our Creating, installing, and running your Warclight application documentation.

Warclight is designed to work with web archive data that is indexed via the UK Web Archive's webarchive-discovery project. A guide on indexing is available here.

Warclight development uses solr_wrapper and engine_cart to host development instances of Solr and Rails server on your local machine.

Ensure Solr and Rails are not running (ports 8983 and 3000 respectively), then:

$ bundle exec rake

$ bundle exec rake warclight:server

Then visit http://localhost:3000. It will also start a Solr instance on port 8983.

You can also run bin/console for an interactive prompt that will allow you to experiment.

Ensure Docker is installed and configured.

$ docker-compose build
$ docker-compose up

Then visit http://localhost:3000. It will also start a Solr instance on port 8983.

To release a new version:

1. Update the version number in lib/warclight/version.rb
2. Run bundle exec rake release, which will create a git tag for the version, push git commits and tags, build the gem file (e.g., gem build warclight.gemspec) and push the .gem file to rubygems.org (e.g., gem push warclight-x.y.z.gem).

Bug reports and pull requests are welcome on Warclight -- see CONTRIBUTING.md for details.

The gem is available as open source under the terms of the Apache License, Version 2.0.

This work is primarily supported by the Andrew W. Mellon Foundation. Other financial and in-kind support comes from the Social Sciences and Humanities Research Council, Compute Canada, the Ontario Ministry of Research, Innovation, and Science, York University Libraries, Start Smart Labs, and the Faculty of Arts and David R. Cheriton School of Computer Science at the University of Waterloo.

Any opinions, findings, and conclusions or recommendations expressed are those of the researchers and do not necessarily reflect the views of the sponsors.

This project drew inspiration from the Arclight and UKWA's Shine, and would like to thank those creators and contributors.