Release Notes 1.14.2

Release Notes - 1.14.2 (November 2008)

These are the project wiki Release Notes for the 1.14.2 release.

Release 1.14.2 is a 'micro' release with a number of small bugfixes.

The 1.14.2 release is available at the archive-crawler Sourceforge project.

Notable Changes

WARC 'metadata' records MIME type matches spec (HER-1541)

WARC 'metadata' records are now written with the MIME type matching the format definition in the 0.18 specification, 'application/warc-fields'. (The data format is unchanged, and any 'text/anvl' content written by earlier releases may be interpreted as 'application/warc-fields' content.)

A bottleneck in StatisticsTracker.saveSourceStats has been removed (HER-1553)

If crawling with the 'source-tag-seeds' option enabled, collecting the hosts-per-source reporting information would begin to take an inordinate amount of unnecessary serialization effort several days into a crawl. The redundant work has been skipped, greatly improving performance when this option is enabled. (The option is by default off.)