Joe Roe

Archaeological Information Systems: A Comparison of Field Recording Software (updated 2026)

2026-08-03T00:00:00+00:00

This page provides an up-to-date comparison of available archaeological information systems, including ARCHES, ARK, Field, FAIMS, Ishtar, Kiosk, OpenAtlas, ArcheoBase, Diggit, Intrasis, and MuseumsGIS.

See updates below for information on changes since the page was first published.

What is an ‘archaeological information system’?

I define an ‘archaeological information system’ (AIS) as software specifically developed for the recording, management and publication of field archaeological data, including both excavations and surveys. It is not a widely used term (though I am not the first to use it), but I think usefully captures a particular type of system which is to archaeology what a geographical information system (GIS) is to geospatial applications.

It is worth noting that the most commonly used archaeological information system is a geographic information system (see honourable mentions below). The development of specific software to extend or replace GIS for in archaeological fieldwork is a relatively new phenomenon. In some contexts (e.g. rescue archaeology in particular countries), one or another AISes have become the norm; but in globally the field still seems to prefer GIS- and/or paper-based recording systems. This makes AIS a rather interesting, experimental niche within archaeological research software engineering.

Summary of available archaeological information systems

Software	Developer	Open source	Self-hostable	Price^^*^^	Notable users
ARCHES	🇺🇸 Getty Conservation Institute	✅	✅	Free	UK Historic Environment Registers
ArcheoBase	🇨🇭 AnalyticBase Sàrl	❌	❌	23 CHF per user per month	Swiss cantonal archaeological services
ARK	🇬🇧 L-P Archaeology	✅	✅	Free
Diggit¹	🇬🇧 Diggit Archaeology	❌	❌	1 GBP per context
Field	🇩🇪 DAI	✅	✅	Free	German Archaeological Institute branches
FAIMS	🇦🇺 Macquarie University / CSIRO	✅	✅	Free
Intrasis	🇸🇪 Arkeologerna	❌	✅	21,000 SEK	Swedish & Danish museums
Ishtar	🇫🇷 Iggdrasil	✅	✅	Free
Kiosk	🇺🇸 Brown University	✅	✅	Free
MuseumsGIS²	🇩🇰 Arkæologisk IT	?	?	c. 37,000 DKK per year	Danish museums
OpenAtlas	🇦🇹 Austrian Academy of Sciences	✅	✅	Free

^^*^^ The prices given are those of the software’s ‘standard’ plan. For details of other plans, offers, or add-ons, see the individual sections below.

How to choose an archaeological information system?

In the tables here I have compiled basic metadata and feature information on archaeological information systems that are actively maintained. Metadata includes the licensing and price model, developer, and notable users. The feature comparison includes major points of differentiation between the different software; features that all systems offer, like data input or file export, are not included.

I have paid particular attention to whether the system is free and open source or commercial, and in which case how much. This is because I usually advocate for the use of free and open source software in academic contexts. Since archaeological information systems are used primarily for data management, rather than an analysis, I would not say that a free and open source software is essential in this case. However, is still offers distinct advantages in terms of interoperability and maintainability (allowing a wider community to share the development burden) and, in an educational context, avoiding locking students into proprietary walled gardens early in their career. That said, the current reality is that some of the best systems are commercial.

My assessments below are written from the perspective of a research archaeologist based at a university, on the basis of the information I could find and demo versions available. I have tried to keep up to date with this field over the years out of a theoretical interest. If I were to use any of the systems myself, it would be in a seasonal research-led field projects and in an educational context, where I want to expose (Danish) students in the modern recording systems they are likely to encounter in their future careers. I am not in a position to evaluate their usefulness to larger institutions or in commercial archaeology contexts, nor to trial each system in the field. Some systems I have not been able to try at all; where this is the case, it is noted below.

Feature comparison

Feature	ARCHES	ArcheoBase	ARK	Diggit	Field	FAIMS	Intrasis	Ishtar	Kiosk	MuseumsGIS	OpenAtlas
Mobile app	❌	✅	❌	✅	❌	✅	✅	❌	✅	✅	❌
Offline recording	❓	❌	❌	✅	✅	✅	❓	❌	✅	❓	❌
Harris matrix	❓	✅	✅	❓	✅	❌	❓	✅	✅	❓	❓
Mapping	✅	✅	✅	❌	✅	✅	✅	✅	❓	✅	✅
GIS integration	✅	✅	✅	✅	✅	✅	✅	✅	❓	✅	✅
Public portal	✅	✅	✅	❓	❓	❌	❓	✅	❓	✅	✅

Mobile app: Dedicated application for iOS or Android devices
Offline recording: Ability to record data without internet connectivity
Harris matrix: Built-in tool for generating stratigraphic relationship diagrams
Mapping: In-app capability to input spatial data (points, lines, polygons)
GIS integration: Ability to export to or integrate with external GIS software (QGIS, ArcGIS, etc.)
Public portal: Web interface for public access to project data

Honourable mentions

Apart from dedicated software, archaeologists routinely make their own digital recording systems with off-the-shelf database (e.g. FileMaker, Microsoft Access) or geographic information system (GIS; e.g. QGIS, ArcGIS) software. In recent years, the availability of mobile GIS apps such as QField and ArcGIS Field Maps have made the latter particularly popular solutions.

These DIY approaches are popular for good reasons. They offer flexibility to adapt to specific project needs, lower cost (especially with open source tools), and no vendor lock-in. They also allow archaeologists to leverage existing skills and workflows rather than learning new systems. For small projects or research-led fieldwork, a custom solution can be more practical than adopting a full archaeological information system. For example, I’ve been using a QGIS-based DIY field recording stack for research-led field surveys for nearly a decade and it works well. It’s customised to my specific needs, is free and open source, and integrates seamlessly with my analysis workflows.

However, there are trade-offs. DIY systems require technical expertise to set up and maintain, and there’s no dedicated support when things go wrong. Each project essentially reinvents the wheel, making it harder to share data and workflows between projects or institutions. Still, I’d guess that the combined use of these DIY systems is significantly greater than any software developed specifically for archaeology. But as my focus here is on explicit archaeological information systems, I won’t consider them further.

Free and open source archaeological information systems

ARCHES

ARCHES is a web-based, self-hosted heritage data management platform developed by the Getty Conservation Institute and World Monuments Fund. It was originally designed for inventories of immovable cultural heritage and has been widely adopted by Historic Environment Records in the United Kingdom, which are served by a dedicated version called Arches for HERs. It has also been adopted by projects including Endangered Archaeology in the Middle East and North Africa (EAMENA) at Oxford³ and the Greater London Historic Environment Record.⁴ The system includes GIS functionality. It requires technical expertise to deploy and maintain.

ARCHES is a sophisticated and mature system with significant institutional backing from major organisations. However, it is primarily tailored for heritage management and inventory purposes, and its utility for active field recording remains unclear. The system is also notably complex to deploy and maintain, requiring substantial technical expertise that may be prohibitive even for experienced system administrators without dedicated IT support.

ARK

ARK (Archaeological Recording Kit) is a web-based, self-hosted system developed by L-P Archaeology. It provides an interface for data entry, editing, mapping, and sharing of archaeological project data, and supports web services and data export. It has been used on excavations including the Villa Magna project in Italy⁵ and Portus.⁶

ARK was an early pioneer of web-based archaeological recording and represented an important step forward in the digitisation of field documentation. However, the system has not been actively maintained or updated since 2014, and its underlying technology stack has become dated. The web-based architecture limits its practical utility in remote field locations where reliable internet connectivity cannot be assumed, and the older codebase presents significant barriers to long-term maintainability and future development.

Field

Field (formerly iDAI.field) is a desktop application (with optional sync server) developed by the German Archaeological Institute (DAI) and the GBV Common Library Network. It allows archaeologists to record, share, and store excavation data and images, customise data models, map records, manage types and inventories, and synchronise data. The system comprises a desktop application (Angular/Electron), a mobile app (React Native, in early development), and an optional sync server.

Field represents a very promising and full-featured approach to archaeological data management, backed by a sizeable research software engineering team and the international presence of the German Archaeological Institute. The involvement of DAI branches across multiple countries provides both development capacity and a built-in user base that lends the project considerable institutional weight and long-term viability. It does not yet have a mobile application, which is increasingly seen as essential for modern in-field recording workflows. However, a mobile app is actively under development in React Native.

FAIMS

FAIMS (Field Acquired Information Management Systems) is an open source platform for offline field data collection, originally developed for archaeology but now used across multiple disciplines including geoscience and humanities. The project has an 11-year development history since 2012, with a major rebuild as FAIMS 3.0 released in 2022. It is cross-platform (Android, iOS, and desktop) and offers a self-service notebook designer for customization without coding. Features include offline data capture, mobile GIS, annotations, certainty sliders, multimedia support, and version control. In 2023, the software was rebranded as Fieldmark™ with a commercial hosting option, though the open source version remains freely available.

FAIMS has been used in over 40 projects and demonstrates a long-term commitment to offline-first field recording. However, based on experience with earlier versions, the system can be difficult to set up and the data model prioritises flexibility over out-of-the-box utility for archaeological workflows. The cross-disciplinary nature of the platform is both a strength (allowing adaptation to diverse research needs) and a weakness (requiring more customization than archaeology-specific systems). The institutional backing from Macquarie University, CSIRO, and the Australian Research Data Commons provides stability, and the complete rebuild in FAIMS 3.0 may address earlier usability concerns.

Ishtar

Ishtar is a web-based, self-hosted system developed by the French company Iggdrasil. It manages data from excavation through to museum storage, including context records, finds, administrative files, warehouses, and conservatory treatments. The system has a modular architecture (cartography, underwater archaeology, public portal, etc.) with customisable forms, automatic inventory generation, and multiple user permission levels. First released in 2017, version 4 was released in 2024.

I have not yet had the opportunity to trial this system in practice. Further assessment will be provided once hands-on evaluation has been conducted.

Kiosk

Kiosk is an iPad-based, self-hosted digital field recording platform developed and maintained at Brown University. It is designed to work offline in remote locations, with data synchronisation possible without internet access. It supports excavation and pedestrian survey recording, Harris matrices, ceramic analysis, custom numbering systems, and legacy data integration. It is used by institutions including NYU, University of Chicago, and Leiden University.

Kiosk’s restriction to the iOS ecosystem represents a significant limitation, particularly for a free and open source application. This platform dependency precludes trialling the system on non-Apple devices and may exclude many potential users whose institutions have standardised on Android or mixed-device environments. For a FOSS project, such vendor lock-in is especially unfortunate and runs counter to the principles of accessibility and platform independence that typically characterise open source software.

OpenAtlas

OpenAtlas is a web-based, self-hosted database application developed by the Austrian Centre for Digital Humanities (ACDH), an institute of the Austrian Academy of Sciences, for managing research data across the humanities. It provides customisable forms for recording complex relationships. Features include an interactive Leaflet-based map with PostGIS spatial data, Linked Open Data reference systems, and data integrity checks. It is used by the Österreichisches Archäologisches Institut and other partner projects.⁷

OpenAtlas is a mature and well-established system with strong institutional backing from the Austrian Archaeological Institute. Like ARCHES, however, it appears to be oriented more towards heritage management, inventory control, and long-term data curation than towards the practical demands of active field recording. Its sophisticated data model and emphasis on complex relational structures may be better suited to post-excavation analysis and museum collections management than to the rapid, context-by-context documentation required during excavation.

Commercial archaeological information systems

ArcheoBase

ArcheoBase is a cloud-based SaaS platform developed by the Swiss company AnalyticBase Sàrl. It covers the full chain of archaeological information processing through modular sub-applications and a mobile app for field recording. Available in multiple European languages, it is priced at 23 CHF per user per month with discounts for larger institutions. For universities, AnalyticBase offer an academic plan where students can receive free access so long as faculty have a license.

ArcheoBase is the most full-featured and polished archaeological information system currently available, with an impressive range of features and a modern and professional user interface. It is also one of the few available multilingual systems. However its cloud-based architecture does not appear to permit offline recording, which is a significant limitation for archaeologists working in remote locations. The annual pricing model does not work well in a research context, where fieldwork is seasonal and typically involves large numbers of temporary staff, and will likely be too expensive for many higher education institutions (though the academic plan alleviates this concern in an educational setting). It also raises the concern of vendor lock-in for institutions with long-term projects and/or a commitment to open source.

Diggit

Diggit is a cloud-based SaaS recording system developed by Diggit Archaeology (UK). It consists of a mobile app (diggitmobile) for on-site recording and a web interface (diggitweb) for exploration, analysis, and export. The interface includes over 240 archaeology-specific icons and charges per context recorded (1 GBP). It is currently available only in the UK, Ireland, and Hong Kong.

Diggit employs an interesting and unconventional pricing model based on the number of contexts recorded rather than the number of users. While this could be advantageous for small projects with limited recording needs, it makes costs difficult to predict and budget for larger excavations where the volume of contexts is uncertain. The system is currently available only in the United Kingdom, Ireland, and Hong Kong, making it inaccessible for evaluation in other regional contexts. As it is not available in Denmark, I was unable to conduct a trial assessment of its capabilities and workflow.

Intrasis

Intrasis is a desktop application (with optional cloud sync) originally designed by Arkeologerna (The Archaeologists) at the Swedish National Historical Museums agency. It combines a database with geographical information for documenting, visualising, and analysing excavation data. While proprietary, it is self-hostable and offers both desktop and mobile apps. According to their website, “it additionally requires an ArcGIS license with minimum licensing to ArcGIS engine runtime (included in ArcGIS desktop).”⁸ It is used by Swedish and Danish museums. The latest version (4.2) runs on PostgreSQL and includes 3D visualisation capabilities.

Intrasis has a rather complex pricing model with various licence types and add-ons. A single-use licence costs 21,000 SEK as a one-time fee, while educational packages are available at 10,000 SEK per year. For full details on licensing options, see the Intrasis pricing page.

Intrasis is one of the longest-established archaeological information systems and has achieved significant adoption in commercial archaeology in Sweden and, to a lesser extent, within the Danish museum system. However, the system’s hard dependency on proprietary ESRI ArcGIS software represents a fundamental barrier to adoption for many users. At a time when open source GIS solutions such as QGIS have become mature, widely-used, and well-supported within the archaeological community, requiring users to maintain expensive proprietary GIS licenses alongside Intrasis is increasingly difficult to justify.

MuseumsGIS

MuseumsGIS is a web-based system with mobile app developed by Arkæologisk IT at Moesgaard Museum, Denmark. It is currently only available to Danish museums. It is built on free and open source software but requires a licence to use, costing approximately 37,000 DKK per year. It comprises a project presentation website (dynamically pulling excavation data including 3D models, historical maps, and geophysics), a mobile field registration app, and QGIS plugins including the FeltTegn tool for converting GPS data to archaeological features. Data is stored on a central database server with full version history at the object level, and is immediately available to both mobile devices and QGIS.

MuseumsGIS is relatively new with an impressive feature set that addresses many of the practical requirements of modern archaeological recording. It is currently tightly tailored to the Danish museum sector in its design and pricing; but the fact that it is built on an open source technology stack means that, in principle, the system could be forked and adapted by other institutions or communities.

Updates

I will keep this page updated as often as possible. Please feel free to contact me with updates, corrections, or suggested additions.

2026-08-19: added information on ArcheoBase’s academic pricing plan
2026-08-03: first version

References

Diggit is only available in the United Kingdom, Ireland, and Hong Kong. ↩
MuseumsGIS is only available in Denmark. ↩
EAMENA database ↩
GLHER ↩
Villa Magna ↩
Portus ↩
THANADOS ↩
Intrasis - What is Intrasis? ↩

era 0.6.0: chronological comparison and difference

2026-07-30T00:00:00+00:00

era v0.6.0 is now available on CRAN:

install.packages("era")
library(era)

This minor release adds functions for chronological comparison of year vectors:

x <- yr(c(200, 100, 300), "BCE")
yr_earlier_than(x, yr(150, "BCE"))
#> # BCE years <yr_lgl[3]>:
#> [1] FALSE FALSE  TRUE
#> # Era: Before Common Era (BCE): Gregorian years (365.2425 days), counted backwards from 1

yr_later_than(x, yr(150, "BCE"))
#> # BCE years <yr_lgl[3]>:
#> [1]  TRUE FALSE FALSE
#> # Era: Before Common Era (BCE): Gregorian years (365.2425 days), counted backwards from 1

And for calculating chronological differences between year vectors:

yr_difference(yr(300, "BCE"), yr(100, "BCE"))
#> # years <yr[1]>:
#> [1] 200
#> # Era: Gregorian years (365.2425 days)

These functions are aware of era directionality: “BCE” years count backwards, so 300 BCE is earlier than 100 BCE. They support work on tempo, a new package I am working on that represents chronological intervals and the temporal relations between them.

controller: tidy messy terminology in R with controlled vocabularies

2026-07-23T00:00:00+00:00

controller is an R package for working with controlled vocabularies. It’s first release (v0.1.0) is now available now on CRAN.

The package addresses something I find myself doing very often in analysis code: tidying messy and inconsistent terminologies. For smaller datasets, dplyr::recode() is okay for this, but writing the mapping out as an R function call gets tedious fast when dealing with a long list of terms. It becomes very tedious when you have variants distinguished only by things like capitalisation (OxA- vs. oxa-), word boundaries (Çatalhöyük vs. Çatal Höyük) or character encoding (ʿAin Ghazal vs. ʽAyn Ghazal).

controller instead defines preferred terms and their variants in a data frame. Its control() verb is the equivalent of dplyr::recode() but using this thesaurus and with a few extra bells and whistles for fuzzy matching and reporting what was (and wasn’t) changed:

library(controller)
data("colour_thesaurus")

shades <- c("daffodil", "purple", "magenta", "azure", "navy", "violet")
control(shades, colour_thesaurus)
#> Replaced values:
#> ℹ daffodil → yellow
#> ℹ azure → blue
#> ℹ navy → blue
#> ℹ violet → purple
#> Warning: Some values of `x` were not matched in `thesaurus`:
#> ✖ magenta

Fuzzy matching means we don’t need to exhaustively list those variants from things like differences in case, word boundaries, or character encoding:

control_ci(toupper(shades), colour_thesaurus)
#> Replaced values:
#> ℹ DAFFODIL → yellow
#> ℹ PURPLE → purple
#> ℹ AZURE → blue
#> ℹ NAVY → blue
#> ℹ VIOLET → purple
#> [1] "yellow"  "purple"  "MAGENTA" "blue"    "blue"    "purple"
#> Warning message:
#> Some values of `x` were not matched in `thesaurus`:
#> ✖ MAGENTA

This package has been hanging around for a while! It started off as a helper function I used for cleaning up site names from prehistoric sites in Southwest Asia. The basic idea was inspired by similar functions that used to exist in c14bazAAR for cleaning sample metadata for radiocarbon date, that I thought were quite neat. So when the maintainers of that package decided to deprecate those, I took over the thesauri as part of c14 and spun the supporting functions off into controller as a standalone package. Then over the years it acquired some more functionality for working with controlled vocabularies (a surprising gap in the R ecosystem), like reading heritage vocabularies in Historic England’s FISH format. Five years later, I am finally getting around to releasing it on CRAN because I need to release c14 on CRAN, because that’s used in analyses I’m now publishing. It’s the research software engineering of changing a lightbulb, basically.

The first release of controller includes:

control(), control_ci(), and control_fuzzy() for recoding values
control_names(), control_names_ci(), and control_names_fuzzy() for recoding names
control_matches() for inspecting how matches were made
read_fish() for reading vocabularies in Historic England’s FISH format
colour_thesaurus, an example dataset

You can install it from CRAN:

install.packages("controller")

Or the development version from GitHub:

remotes::install_github("joeroe/controller")

You can find the full documentation at https://controller.joeroe.io.

When was the plough invented?

2025-08-24T00:00:00+00:00

The invention of the ox-drawn plough was a turning point in the prehistory of agriculture because it was the first time food production was decoupled from human labour. This is why, from Medieval Europe to the ancient Near East farms were measured by the number of oxen (later horses etc.) working it, not people.

Yet it’s surprisingly difficult to pinpoint when this actually happened.

Archaeological sites with evidence for early ploughing in Europe (van Willigen et al. 2024, fig. 1, CC BY)

The earliest direct evidence we have for ploughing are marks made in ancient sediment, though these are very rarely preserved. In Europe the oldest known are from Switzerland (5000 BCE), Denmark (4000 BCE) and Italy (4300 BCE, at Saint-Martin-de-Corléans). In each case the date coincidences with, or is shortly after, the arrival of agriculture in that region. This implies that ploughing was part of European farming from the beginning. In other words, like agriculture itself, it must have been brought there from the Middle East.

Ploughmarks from Anciens Arsenaux in Switzerland – the earliest known direct evidence for ploughing (van Willigen et al. 2024, fig. 3, CC BY)

Unfortunately we haven’t (yet) found the right conditions for preserved ploughmarks in the Middle East. Instead, archaeologists have relied on indirect evidence of ploughing. The main source of evidence is cattle bones, which can display work-related injuries or signs of castration (oxen are usually castrated to make them easier to work with). If we accept this evidence, ploughing could have been invented as early as 8500 BCE – right after cattle were domesticated.

There are some problems, though.

The pathologies used to identify working animals are rare in the archaeological record and can also be caused by illness. Castrates are distinguished by comparing measurements of their bones to modern animals, but we can’t be sure this is a valid comparison. At this early stage cattle were still being domesticated and probably were in contact with wild auroch populations, so they likely displayed more morphological variation than modern domestic animals. Some experts even dispute the fact that cattle were domesticated by 8500 BCE. instead putting it up to a thousand years later. As with working, we largely rely on indirect evidence for cattle domestication, so it is hard to pinpoint with confidence.

So where does this leave us? Based on the evidence from Europe, ploughing was invented before 5000 BCE, but could go as far back 7500–8500 BCE in the Middle East. In other words, we don’t know if the plough was a late addition to prehistoric agriculture—adopted somewhere on the way from the Middle East to Europe—or if it was an integral part of it from the start.

Spread of farming in Western Eurasia (Gronenborn et al. 2023, CC BY)

This is a big problem for our understanding of human social and economic prehistory. We think the earliest farmers practised a sort of ‘garden agriculture’ based on limited tilling with hand tools. Ploughing up whole fields to (presumably) sow them with monocultures is a another beast entirely, ecologically speaking. Animal labour is also what made it possible for one person to reap the harvest of more land than they themselves could physically work – or compel other people to do it for them!

Was the potential for proto-serfdom and cattle-powered ecocide inherent in agriculture from the beginning? Or did our farming ancestors spend as much as three thousand years as essentially egalitarian gardeners before things took a turn for the worse? That’s the difference dating the plough makes!

Adapted from a Mastodon thread posted to archaeo.social on 7 August 2025.

era 0.5.0: chronological ordering and extremes

2024-11-20T00:00:00+00:00

era v0.5.0 is now available on CRAN:

install.packages("era")

This minor release adds functions for chronological ordering (yr_sort()) of year vectors:

# Forward-counting era:
x <- yr(c(200, 100, 300), "CE")
yr_earliest(x)
#> # CE years <yr[1]>:
#> [1] 100
#> # Era: Common Era (CE): Gregorian years (365.2425 days), counted forwards from 0
yr_latest(x)
#> # CE years <yr[1]>:
#> [1] 300
#> # Era: Common Era (CE): Gregorian years (365.2425 days), counted forwards from 0
yr_range(x)
#> # CE years <yr[2]>:
#> [1] 100 300
#> # Era: Common Era (CE): Gregorian years (365.2425 days), counted forwards from 0

# Backward-counting era:
y <- yr(c(200, 100, 300), "BCE")
yr_earliest(y)
#> # BCE years <yr[1]>:
#> [1] 300
#> # Era: Before Common Era (BCE): Gregorian years (365.2425 days), counted backwards from 1
yr_latest(y)
#> # BCE years <yr[1]>:
#> [1] 100
#> # Era: Before Common Era (BCE): Gregorian years (365.2425 days), counted backwards from 1
yr_range(x)
#> # CE years <yr[2]>:
#> [1] 100 300
#> # Era: Common Era (CE): Gregorian years (365.2425 days), counted forwards from 0

And for calculating their extreme values (yr_earliest(), yr_latest(), and yr_range())

# Forward-counting era:
x <- yr(c(200, 100, 300), "CE")
yr_earliest(x)
#> # CE years <yr[1]>:
#> [1] 100
#> # Era: Common Era (CE): Gregorian years (365.2425 days), counted forwards from 0
yr_latest(x)
#> # CE years <yr[1]>:
#> [1] 300
#> # Era: Common Era (CE): Gregorian years (365.2425 days), counted forwards from 0
yr_range(x)
#> # CE years <yr[2]>:
#> [1] 100 300
#> # Era: Common Era (CE): Gregorian years (365.2425 days), counted forwards from 0

# Backward-counting era:
y <- yr(c(200, 100, 300), "BCE")
yr_earliest(y)
#> # BCE years <yr[1]>:
#> [1] 300
#> # Era: Before Common Era (BCE): Gregorian years (365.2425 days), counted backwards from 1
yr_latest(y)
#> # BCE years <yr[1]>:
#> [1] 100
#> # Era: Before Common Era (BCE): Gregorian years (365.2425 days), counted backwards from 1
yr_range(x)
#> # CE years <yr[2]>:
#> [1] 100 300
#> # Era: Common Era (CE): Gregorian years (365.2425 days), counted forwards from 0

Essentially these are all wrappers for base functions (sort(), min(), max() and range()) that are aware of the directionality of the era system attached to the vector: “CE” years are counted forwards, “BCE” years are counted backwards, etc. I decided to implement them as prefixed functions instead of S3 methods for yr vectors because I didn’t want to suprise people when they used e.g. max() expecting the numerical maximum and got the chronologically latest value instead.

Deploy a Fly app with Woodpecker CI

2024-01-09T00:00:00+00:00

This post describes how to set up continuous deployment to fly.io using Woodpecker, an open source continuous integration engine. The Fly documentation includes instructions on how to set up continuous deployment with GitHub Actions and with GitLab CI/CD, but I couldn’t find anything for Woodpecker. Fortunately, the process is almost identical to setting it up with GitLab CI/CD, so I could figure it out by following along that tutorial and making minor changes.

My use case was to deploy from a Codeberg repository, but the instructions below should also work for any forge supported by Woodpecker (currently GitHub, Gitea, Forgejo, GitLab, and Bitbucket).

Connect your repository to Woodpecker

First things first, you’ll need access to a Woodpecker instance. Codeberg users can request access to a hosted version, as long as you intend to use it for public, freely-licensed code. Otherwise you can self-host it. Woodpecker come with both a web interface and a command line interface. You can connect the CLI to your Woodpecker instance using the personal access token displayed under User Settings → API.

Once you have Woodpecker up and running, activate it for the repository that you want to deploy, i.e. the one that contains your fly.toml configuration file. Be sure to disable ‘Allow Pull Requests’ in the project settings, otherwise anybody who can make a pull request will have access to your Fly machines!

Create a workflow file

Create .woodpecker/deploy.yml in your repository with the following contents:

steps:
  deploy:
    image: golang
    commands:
      - apt-get update -qq && apt-get install -y curl
      - curl -L https://fly.io/install.sh | sh
      - /root/.fly/bin/flyctl deploy
    secrets: [ fly_access_token ]

This sets up a ‘deploy’ workflow which downloads a shell script to install fly, then runs flyctl deploy. The choice of base image is arbitrary since all we really need installed is curl and bash.

Set up a Fly access token

In order to deploy securely from the container created by Woodpecker, we need to authenticate with Fly in such a way that the credentials do not leak into the (public) CI logs. To do this, we will generate a unique access token and pass it into the container as an environment variable using Woodpecker’s secrets store.

First, generate a new deploy token using the Fly web interface or command line:

flyctl tokens create deploy -n woodpecker

It’s a good idea to use a dedicated token for woodpecker deployments, so if you need to you can revoke it without affecting any other integrations you might have. You might also want to set a shorter expiry time with the -x option (the default is twenty years). Copy or save your newly generated secret somewhere now, because it’s only shown once.

Next, create a secret named fly_access_token that contains your access token. You can do this using the Woodpecker web interface or the command line:

woodpecker secret add -name fly_access_token -event push -event manual -repository <your_repository> -value '<your_access_token>'

Note that we only make the secret available when the workflow is triggered by a push from an authorised contributor to the repository or run manually by somebody with access to Woodpecker CI. Your usage might vary, but you probably should not make it available on pull requests, otherwise a careless or malicious pull request could leak your access token.

Deploy

That’s it! When you push these changes to your remote repository, Woodpecker should immediately deploy to Fly. You can monitor the status of the deployment in the Woodpecker CI web interface. In the Woodpecker repository settings, under Badge, you’ll also find a badge that you can add to your README to show the status of the last deployment.

rpaleoclim v1.0.0: paleoclimate data in R

2023-05-02T00:00:00+00:00

PaleoClim (Brown et al. 2018, Scientific Data) is a set of high-resolution paleoclimate surfaces covering the whole world. The data is derived from HadCM3, one of the major ‘general circulation models’ that is used to forecast climate change, turned backwards to ‘predict’ conditions for key climate periods in the past. This is then ‘downscaled’ to a high spatial resolution (up to 2.5 minutes) using modern climate data. A continuous set of reconstructions from the near-present to the Last Glacial Maximum (c. 0.3–21 ka, divided into 7 periods) are available, plus snapshots for the Last Interglacial (c. 130 ka), MIS19 (c. 787 ka), mid-Pliocene (c. 3.205 Ma and 3.3 Ma). For convenience, modern data from CHELSA (used for downscaling) is also bundled in the same format.

PaleoClim is my go-to paleoclimate dataset for modelling prehistoric environments because as far as I know it has the best spatial resolution available for my main period of interest, the Late Pleistocene and Early Holocene.¹ It’s also generally more accessible and easier to use than ‘raw’ GCM predictions and the choice of time slices is a good fit for prehistoric archaeology.² Its API is very simple: from the PaleoClim website you can download archives for each period at a number of resolutions, with each archive including a set of GeoTIFFs representing the 19 bioclimatic variables widely used in ecological modelling.

Alternatively, rpaleoclim is a simple R package that automates the process of downloading, reading and cropping PaleoClim data. It’s been availble on GitHub for a few years, but the latest release v1.0.0 adds support for terra-format rasters and also brings the package to CRAN. This means you can now install it easily with:

install.packages("rpaleoclim")

The way I usually use rpaleoclim is to get the reconstructions for a particular region for a set of periods. For example, Europe in the Late Holocene:

library("rpaleoclim")
library("terra")

europe <- c(-15, 45, 30, 90)
europe_lh <- paleoclim("lh", "10m", region = europe)

plot(europe_lh[["bio_12"]], main = "total annual precipitation")

Further functionality is explained in the introduction to rpaleoclim vignette. One thing I’d highlight is the caching facility, which my default uses a temporary directory to store downloaded PaleoClim for the length of the R session. To keep things reproducible, I recommend using the cache_path argument to use a location within your project directory instead.³ That way, if the online versions of the PastClim data ever disappears or moves, the dataset you rely on will still be there – something I’ve been caught out by many, many times!

Please note that rpaleoclim simply provides a convenient way to use the PaleoClim dataset in R; I don’t claim any credit for the actual data. If you use it in a publication, the PaleoClim authors request that you cite both their derived dataset (i.e. PaleoClim, Brown et al. 2018) and the original climatologies used. These references are included in the package:

citation("rpaleoclim")

## Please cite both PaleoClim and original datasets used.
## 
## PaleoClim and dataset 'mis19':
## 
##   Brown Jason L, et al. "PaleoClim, high spatial resolution
##   paleoclimate surfaces for global land areas". Scientific Data 5.
##   (2018): 180254.
## 
## Datasets 'lh', 'mh', 'eh', 'yds', 'ba' and 'hs1':
## 
##   Fordham Damien A, et al. "PaleoView: a tool for generating continuous
##   climate projections spanning the last 21 000 years at regional and
##   global scales". Ecography 40. (2017): 1348-1358.
## 
## Dataset 'lig':
## 
##   Otto-Bliesner Bette L., et al. "Simulating Arctic Climate Warmth and
##   Icefield Retreat in the Last Interglaciation". Science 311. (2006):
##   1751-1753.
## 
## Dataset 'mpwp':
## 
##   Hill Daniel J. "The non-analogue nature of Pliocene temperature
##   gradients ". Earth and Planetary Science Letters 425. (2015):
##   232-241.
## 
## Dataset 'm2':
## 
##   Dolan Aisling M, et al. "Modelling the enigmatic Late Pliocene
##   Glacial Event — Marine Isotope Stage M2". Global and Planetary Change
##   128. (2015): 47-60.
## 
## Dataset 'cur' and 'lgm' (CHELSA):
## 
##   Karger Dirk Nikolaus, et al. "Climatologies at high resolution for
##   the earth’s land surface areas". Scientific Data 4. (2017): 170122.
## 
## To see these entries in BibTeX format, use 'print(<citation>,
## bibtex=TRUE)', 'toBibtex(.)', or set
## 'options(citation.bibtex.max=999)'.

Last year another package, pastclim was released offering similar functionality to rpaleoclim. It includes two datasets—Beyer et al. 2020 and Krapp et al. 2021—that considerably extends the range of reconstructions, as well as providing functions for processing new datasets in netCDF format. Unfortunately the spatial resolution of these datasets is quite coarse compared to PaleoClim (0.5°), so they’re not so useful for my purposes. So I’ll be keeping an eye on pastclim, but for the time being at least I intend to continue developing rpaleoclim. I would like to add support for complementary datasets like StableClim. I’m also unsure about whether to stick with terra for raster data or move to stars – any input on this is very welcome!

Notes

I could well be wrong. It’s been a few years since I actually looked into it. ↩
Even better would be PaleoView, which offers decadal temporal resolution over the last 21,000 years. Unfortunately, I could never get its Python-based frontend to run on Linux. ↩
Following the rrtools conventions, I use analyis/data/derived_data. ↩

What did the first farmers actually farm?

2023-04-12T00:00:00+00:00

Eight so-called founder crops—emmer wheat, einkorn wheat, barley, lentil, pea, chickpea, bitter vetch, and flax—have long been thought to have been the bedrock of Neolithic economies. But early prehistoric sites in Southwest Asia keep turning up more ‘exotic’ (to us) species, like the sedge tubers mixed into the earliest known breads or the new glume wheat, an important ancient crop that’s now extinct.

Illustrations of the eight founder crops. Source: Wikipedia.

Is it a coincidence that the founder crops are all species that remains staples today? In our new paper, Amaia Arranz-Otaegui and I examine what plants actually turn up the most in the remains of the earliest agricultural societies. We gathered data on 144 million botanical remains from 135 prehistoric sites across Southwest Asia. Our analysis included the Neolithic (c. 9700 to 4500 BCE) and the periods immediately before and after – 10,000 years of plant exploitation. We found that Neolithic economies were much more diverse than previously thought, incorporating dozens of species of cereals, legumes, small-seeded grasses, brassicas, pseudocereals, sedges, flowering plants, trees, and shrubs. Free-threshing wheat, grass pea, faba bean, and ‘new’ glume wheat were especially widely cultivated.

Cross-assemblage abundance of the most common plant taxa (left) and ubiquity and abundance of the founder crops (right) through the Neolithic. Source: Arranz-Otaegui & Roe 2023, figs. 3 and 4.

It turns out that the ‘founder crops’ weren’t particularly special at all: rarely cultivated for the first thousand years of the Neolithic, they were never ubiquitous, weren’t the first species to be domesticated, or the first or only to be spread outside of Southwest Asia. They did become more common over the course of the Neolithic but, remarkably, this can be explained almost entirely by a steadily increasing reliance on wheat as a staple crop. There’s a story waiting to be told there, I’m sure! For us, it no longer makes sense to talk about crop packages. Neolithic farmers cultivated, managed and otherwise experimented with a huge diversity of plants. Tracing the individual trajectories of these crops in the accumulated archaeobotanical remains reveals many fascinating new stories.

For more details, see our new open access paper in Vegetation History & Archaeobotany. The data and R code supporting the analysis is available on GitHub and Zenodo.

Graham Hancock is right about pseudoarchaeology on Wikipedia

2022-11-14T00:00:00+00:00

In a recent interview on a popular podcast, pseudoarchaeologist Graham Hancock had this to say about Wikipedia:

[T]he point about Wikipedia is, that’s the first place, when somebody hears my name or hears about my ideas, first place they gonna go have a look is Wikipedia. And immediately they’re gonna get turned off.

And he’s absolutely right, of course. If you’re itching to debunk the racist garbage he spreads in Netflix’s new series Ancient Apocalypse, consider editing Wikipedia. It really makes a difference.

Hancock goes on to say that “you can’t edit my Wikipedia page, they’ve locked it, and it’s controlled by a group of academics”, whichis not true. Anybody with an account older than four days can edit the article about him. And anyone can edit the articles about the archaeological sites he misrepresents, with or without an account. Hancock fans regularly do try to distort and whitewash these pages, and the small group of volunteer who edit archaeology-related topics are struggling to stay on top of it. Help from academics and other people with subject-matter expertise is really, really appreciated.

The article on Gunung Padang, an Indonesian archaeological site featured in the first episode of the series, has almost no reliably-sourced information on the real archaeology there! Two other sites Hancock tells tales about are Għar Dalam in Malta and Derinkuyu in Turkey. These articles are in an okay state, but so short that a viewer might think he’s right when he says archaeologists are ignoring them. Or there’s Göbekli Tepe, a favourite of many pseudoarchaeologists (it “changes everything!”). There are featured articles on the site in the German and French Wikipedias, but the English one is underdeveloped. So translators could really help here.

I think editing Wikipedia is the most impactful thing we can do for the public understanding of archaeology. But unfortunately, it is sometimes easier said than done. If prefer not to edit directly, writing responses and rebuttals that Wikipedians can cite is also a huge help!

Addendum (2022-11-17): The chart below shows the daily pageviews of #Wikipedia’s article on Göbekli Tepe over the last year – spot the Ancient Apocalypse effect! It’s even more stark for Gunung Padang, a site that was apparently all but unknown to a general audience before Graham Hancock featured it in the first episode of his pseudoarchaeology documentary.

Daily page views for the English Wikipedia articles on Göbekli Tepe (left) and Gunang Padang (right).

Adapted from the original Twitter thread on 2023-04-13.

Wikipedia is the best way to counter pseudoarchaeology

2022-01-12T00:00:00+00:00

I was glad to see a brief nod to @Wikipedia in a recent Input article on pseudoarchaeology in social media. Its importance in countering pseudoarchaeology and disinformation about the past is often neglected.

Ask yourself: what do you do when you encounter a new idea and want to check it or read about it further? For most people, the answer is “look it up on Wikipedia”. Editing it is one of the most effective public engagement tools out there, and has almost no barrier to entry. Pseudoarchaeologists know this very well. They relentlessly try to add their newest theories to articles, rewrite biographies of advocates to remove criticism and make them sound more legitimate, and ‘edit war’ with regular editors to keep their changes.

On the other side you have a small group of volunteer editors, almost all not professional archaeologists, fighting to keep articles in line with mainstream science. It can seem like a thankless task, but it works. To take a petty-but-satisfying example from the Input article: “When Graham Hancock’s Wikipedia page was edited in 2019 to include references to him engaging in pseudoarchaeology, it set off a stream of furious commentary […] calling it ‘attempted character assassination.’”

But Wikipedia editors rely on academics to give them the ammunition for this battle: “editors look to the consensus of professional archaeologists in deciding whether to label something ‘pseudoarchaeology’ or list it as a credible theory”. verything on Wikipedia must be cited to ‘reliable sources’ (e.g. journal articles). If there are no critical reliable sources, then its editors cannot fight pseudoarchaeology effectively. And they cannot create those sources themselves: that’s the job of archaeologists. This is why the widespread belief that “giving alternative theories professional attention only serves to legitimize them” is so frustrating. As an archaeologist, you might know that the latest ‘alternative theory’ is rubbish. But others probably can’t spot that without help. Writing rebuttals and critiques to pseudoarchaeology might seem like a waste of time, but if archaeologists don’t use make their knowledge available to Wikipedians, there is no way that Wikipedians can make it available to the public.

As an archaeologist, you don’t need a big social media following to effectively combat pseudoarchaeology. Just use the tools that are available to you—your privilaged ability to produce ‘reliable sources’ in your area of expertise—and Wikipedia editors will do the rest.