NeoNet
Thomas Huet, Niccolò Mazzucco, Miriam Cubas Morera, Juan Gibaja, F. Xavier Oms, António Faustino Carvalho, Ana Catarina Basilio, Elías López-Romero
NeoNet serves as a framework to study the transition from the Late Mesolithic to the Early Neolithic through spatio-temporal modeling of radiocarbon dates and by offering:
- curated datasets
- R functions and reference data
- interactive app
Neonet radiocarbon datasets: "NeoNet Med" (North-Central Mediterranean, location) and "NeoNet Atl" (European South Atlantic river basin, location), here filtered by periods.
Studied periods are listed in the periods.tsv table:
period |
period_full_name |
color |
EM |
Early Mesolithic |
|
MM |
Middle Mesolithic |
|
LM |
Late Mesolithic |
|
LMEN |
Late Mesolithic/Early Neolithic |
|
UM |
Undefined Mesolithic |
|
EN |
Early Neolithic |
|
EMN |
Early/Middle Neolithic |
|
MN |
Middle Neolithic |
|
LN |
Late Neolithic |
|
UN |
Undefined Neolithic |
|
NeoNet datasets
| Med | Atl |
|------ |----- |
| [id00140_doc_elencoc14.tsv](https://github.com/zoometh/neonet/blob/main/inst/extdata/id00140_doc_elencoc14.tsv) | [id00164_doc_elencoc14.tsv](https://github.com/zoometh/neonet/blob/main/inst/extdata/id00164_doc_elencoc14.tsv) |
|
|
|
These datasets are harvested by the c14bazAAR package, functions get_c14data("neonet")
and get_c14data("neonetatl")
NeoNet app
mapping the Late Mesolithic/Early Neolithic transition
NeoNet app is an R Shiny application for mapping radiocarbon (C14). The application offers a mobile geographic window for date selection by location, various filters on chronology and date quality, a calibration window, and other tools to create a user-friendly interface supported by curated datasets of radiocarbon dates and archaeological contexts. NeoNet app is hosted on the server of the University of Pisa. This NeoNet app uses this radiocarbon dataset: https://doi.org/10.13131/archelogicadata-yb11-yb66 published as a data paper in the Journal of Open Archaeology Data and describe in this web document.
NeoNet app
NeoNet app dev
TODO
- On export action (download button), export this metadata:
- max SD
- selected periods
- dataset (Med, Atl) DOI
- timestamp
- bibliographical reference
NeoNet functions
radiocarbon management
NeoNet functions enable the handling of radiocarbon dates sourced from the dataset or exported from the interactive app. Current functions cover:
Data
Reference data
Data preparation
Starting by running these neo_*()
functions to manage a new XLSX dataset. Sourcing functions:
source("R/neo_subset.R")
source("R/neo_bib.R")
source("R/neo_matlife.R")
source("R/neo_calib.R")
source("R/neo_merge.R")
source("R/neo_html.R")
source("R/neo_datamiss.R")
source("R/neo_datasum.R")
source("R/neo_doi.R")
Read the new dataset and bibliographic file
data.c14 <- paste0(getwd(), "/inst/extdata/", "NeoNet_atl_ELR (1).xlsx")
df.bib <- paste0(getwd(), "/inst/extdata/", "NeoNet_atl_ELR.bib")
Cleaning the dataset and making it conform to the published NeoNet dataset
df.c14 <- openxlsx::read.xlsx(data.c14)
df.c14 <- neo_subset(df.c14,
rm.C14Age = TRUE,
rm.Spatial = FALSE,
rm.Period = FALSE)
df.c14 <- neo_calib(df.c14)
neo_doi(df.c14)
Prepare the dataset for the Shiny application by merging it with NeoNet Med, calculating materil life duration, and HTML popup layouts
df.c14 <- neo_merge(df.c14 = df.c14,
data.bib = data.bib,
merge.bib = F)
df.c14 <- neo_matlife(df.c14)
df.c14 <- neo_html(df.c14)
Export the merged dataset
write.table(df.c14, "C:/Rprojects/neonet/R/app-dev/c14_dataset_med_x_atl.tsv",
sep = "\t",
row.names = FALSE)
Data report
Calculate basic statistics
Calculate basic statistics: missing data
<img alt="img-name" src="https://raw.githubusercontent.com/zoometh/neonet/main/results/figure3.jpg"
" width="700">
Missing data (empty cells)
Data alignment
Retrieve dates coming from other databases (with the c14bazAAR R package) and mapped to be compliant with the Neonet format (…, LM, EN, …) and functions, using neo_dbs_parse()
, a mapping table (XLSX) created with neo_dbs_create_ref()
, and neo_dbs_align()
:
source("R/neo_dbs_parse.R")
source("R/neo_dbs_align.R")
# spatial and chronological limits
where <- sf::st_read("https://raw.githubusercontent.com/zoometh/neonet/main/doc/talks/2024-simep/roi.geojson", quiet = TRUE)
when <- c(-9000, -4000) # in cal BC
# parameters
l.dbs <- c("calpal", "medafricarbon", "agrichange", "bda", "calpal", "radon", "katsianis")
col.c14baz <- c("sourcedb", "site", "labnr", "c14age", "c14std", "period", "culture", "lon", "lat")
# collect the dates form different DBs, standardize the cultural period layout, filter on 'when' and 'where'
df <- neo_dbs_parse(l.dbs = l.dbs,
chr.interval.uncalBC = when,
roi = where,
col.c14baz = col.c14baz)
df.c14 <- neo_dbs_align(df,
mapping.file = "C:/Rprojects/neonet/doc/ref_table_per.xlsx")
Alignments are done using the mapping file ref_table_per.xlsx, a default parameter in neo_dbs_align()
.
Mapping file
The XLSX file ref_table_per.xlsx has been created using:
df <- neo_dbs_parse(l.dbs = l.dbs,
chr.interval.uncalBC = when,
roi = where,
col.c14baz = col.c14baz)
# create the mapping file
neo_dbs_create_ref(df.all.res = df,
root.path = "C:/Rprojects/neonet/results",
outFile = "df_ref_per.xlsx")
This mapping file ref_table_per.xlsx is a reference table to map cultural assessment coming from external DBs, collected throug c14bazAAR, to the Neonet one (…, LM, EN, …) format. This XLSX file has to be updated manually by specialists.
<img alt="img-name" src="https://raw.githubusercontent.com/zoometh/neonet/main/doc/img/ref_table_per.png"
" width="600">
Equivalences between third part databases cultural assessment (columns 'period' and 'culture') and Neonet classes (column 'class')
For example:
df.c14 <- neo_dbs_align(df,
mapping.file = "C:/Rprojects/neonet/doc/ref_table_per.xlsx")
head(df.c14)
Gives a dataframe where all fields have been renamed to be parsed with the Neonet functions. Among this mapping the column ‘Period’ with, for example, MM
Middle Mesolithic) maps the bda
period = Mésolithique 1
and culture = Capsien ancien
or Capsien typique
(columns ‘db_period’ and ‘db_culture’):
sourcedb |
SiteName |
LabCode |
C14Age |
C14SD |
db_period |
db_culture |
Period |
lon |
lat |
bda |
Mechta el Arbi |
Poz-92231 |
6600 |
80 |
Mésolithique 1 |
Capsien ancien |
MM |
6.130900 |
36.09940 |
bda |
Mechta el Arbi |
Poz-92232 |
6250 |
130 |
Mésolithique 1 |
Capsien ancien |
MM |
6.130900 |
36.09940 |
bda |
Mechta el Arbi |
Poz-92230 |
3500 |
120 |
Mésolithique 1 |
Capsien ancien |
MM |
6.130900 |
36.09940 |
bda |
Kef Zoura D |
UOC-2925 |
7787 |
48 |
Mésolithique 1 |
Capsien typique |
MM |
7.682121 |
35.04205 |
bda |
Bortal Fakher |
L-240A |
6930 |
200 |
Mésolithique 1 |
Capsien typique |
MM |
8.176087 |
34.35480 |
bda |
Relilaï (B) |
Gif-1714 |
7760 |
180 |
Mésolithique 1 |
Capsien typique |
MM |
7.694694 |
35.04480 |
Outlier dates
To filter aberrant dates, a combination of different function allow to retrieve the current information (once mapped to the Neonet layout) of these potential outliers and their original information (from their source database).
Here the dates 260 (circled in red), 396 and 345 seem aberrant
To check the info of the date numbered `260`, do:
```R
source("R/neo_find_date.R")
source("R/neo_dbs_info_date.R")
abber.date <- neo_find_date(df = isochr$data, print.it = FALSE, idf.dates = 260)
ad <- neo_dbs_info_date(df.c14 = df.c14, LabCode = abber.date$labcode)
```
Gives:
```
[1] "Reads a 'sf' dataframe"
[1] "Layout for 'aberrant dates' format:"
neonetatl Sac-1676 El Retamar -6263 EN None None
```
Where `neonetatl` is the source DB, `Sac-1676` is the labnum, `El Retamar` the site name, `-6263` the weighted mean (cal BC), `EN` the period (Early Neolithic), `None` the culture (no data) and the second `None` a place holder for a description explaining why this date has been discarted.
If the archaeological documentation shows that date `Sac-1676` (*aka* `260`) is wrong (mixed assemblage, possibly coming from another layer, etc.). The last line `neonetatl Sac-1676 El Retamar -6263 EN None None` can be pasted as it into the [c14_aberrant_dates.tsv](https://github.com/zoometh/neonet/blob/main/inst/extdata/c14_aberrant_dates.tsv) file.
More info can be found for this date, reusing the `ad` dataframe. Running the following functions helps to contextualize the date.
```R
source("R/neo_dbs_info_date_src.R")
neo_dbs_info_date_src(db = ad$sourcedb,
LabCode = ad$LabCode)
```
Gives:
```
sourcedb sourcedb_version labnr c14age c14std site feature period material species country lat
1 neonetatl 2023-12-08 Sac-1676 7400 100 El Retamar Conchero 6 EN shell Spain 36.57873
lon shortref
1 -6.226273 Cantillo-Duarte et al. 2010
```
The list of aberrant dates, for example [c14_aberrant_dates.tsv](https://github.com/zoometh/neonet/blob/main/inst/extdata/c14_aberrant_dates.tsv), is used to discard some dates with the `neo_dbs_rm_date()` function
```R
source("R/neo_dbs_rm_date.R")
df_filtered <- neo_dbs_rm_date(df.c14)
```
By default, dates having a dash prefix in their `sourcedb` column will be skipped (ex: `-radon`)
<img alt="img-name" src="https://raw.githubusercontent.com/zoometh/neonet/main/results/image-3.png"
" width="800">
Screenshot of the 'c14_to_remove.tsv' table
A template script to run this operation: [run_outliers.R](https://github.com/zoometh/neonet/blob/main/doc/run_outliers.R)
#### Wrong coordinates
Site coordinates can be wrong, for example for the Jordian site of Sabha ([Xronos](https://xronos.ch/sites/12960) | [Google maps](https://maps.app.goo.gl/r4GEuFyzwysAwLe89)). Curated coordinates are listed in the [c14_corrected_coordinates.tsv](https://github.com/zoometh/neonet/blob/main/inst/extdata/c14_corrected_coordinates.tsv) file and handle by the `neo_dbs_coord_dates()` function.
#### Correct Site names
When aggregate different databases, using ce c14bazAAR package, two same sites can have different site names (ex: Pollera and Grotta della Pollera). To avoid these mismatches, the function `neo_dbs_sitename_dates()` uses the mapping table [c14_corrected_sitenames.tsv](https://raw.githubusercontent.com/zoometh/neonet/main/inst/extdata/c14_corrected_sitenames.tsv)
```R
source("R/neo_dbs_sitename_dates.R")
df.c14 <- neo_dbs_sitename_dates(df.c14)
```
### SPD plot
Plot the summed probabilty densities (SPD) of the two datasets, once `df.c14` calculated. The function `neo_spd()` calls `neo_spdplot()`. The latter has been adapted from `rcarbon::plot.stackCalSPD.R`, to fetch NeoNet default period colors.
```R
library(rcarbon)
source("R/neo_spd.R")
source("R/neo_spdplot.R")
neo_spd(df.c14 = df.c14)
```
<img alt="img-name" src="https://raw.githubusercontent.com/zoometh/neonet/main/doc/img/neonet_med_x_atl_spd.png"
" width="700">
NeoNet dataset SPD with default period colors
`neo_spd()` can be run on a GeoJSON file exported from the NeoNet app (see "export dates" in the [web document](https://zoometh.github.io/neonet/doc/#export_dates). For example [neonet-data-2023-09-24.geojson](https://github.com/zoometh/neonet/blob/main/results/neonet-data-2023-09-24.geojson), see also: [isochrones](https://github.com/zoometh/neonet?tab=readme-ov-file#isochrones)
```R
neo_spd(df.c14 = "https://raw.githubusercontent.com/zoometh/neonet/main/results/neonet-data-2023-09-24.geojson",
export = T)
```
In the same way, SPD can be done for KCC
```R
source("R/neo_spd.R")
source("R/neo_spdplot.R")
source("R/neo_kcc_extract.R")
kcc.file <- c("koppen_6k.tif", "koppen_7k.tif", "koppen_8k.tif",
"koppen_9k.tif", "koppen_10k.tif", "koppen_11k.tif")
df_cc <- neo_kcc_extract(df.c14 = df_filtered,
kcc.file = kcc.file)
neo_spd(df.c14 = df_cc, color.on = "kcc")
```
### Isochrones
Create a map with isochrone contours to model the spread of Neolithic using the `neo_isochr()` function.
#### Example
The file [neonet-data-2023-09-24.geojson](https://github.com/zoometh/neonet/blob/main/results/neonet-data-2023-09-24.geojson) is an export from the NeoNet app (see "export dates" in the [web document](https://zoometh.github.io/neonet/doc/#export_dates)). This GeoJSON file can be curated in a GIS (ex: removing aberrant dates) before running the following functions (`neo_isochr`, `neo_spd`, etc.).
<img alt="img-name" src="https://raw.githubusercontent.com/zoometh/neonet/main/results/neonet-data-2023-09-24-app.png"
" width="700">
Screen capture of the NeoNet app before the export of the `neonet-data-2023-09-24.geojson` file: Early Neolithic (EN) dates only having a SD ≤ 50, and calBC interval between -7000 and -3000
The output is a map with isochrones calculated on the median of calibrated dates.
```R
library(rcarbon)
source("R/neo_isochr.R")
source("R/neo_spd.R")
source("R/neo_calib.R")
neo_isochr(df.c14 = "https://raw.githubusercontent.com/zoometh/neonet/main/results/neonet-data-2023-09-24.geojson",
show.lbl = FALSE)
```
Where `neo_calib()` calculate the cal BC min and max (i.e, calibrates), and the medidan (with 'intcal20' and the [Bchron](https://andrewcparnell.github.io/Bchron/) R package)
<img alt="img-name" src="https://raw.githubusercontent.com/zoometh/neonet/main/results/neonet-data-2023-09-24-isochr.png"
" width="700">
Output map from the `neonet-data-2023-09-24.geojson`
#### Interpolation
Isochrones are created from the interpolation map:
```R
source("R/neo_isochr_inter_map.R")
inter.map <- neo_isochr_inter_map(isochr$inter)
inter.map
## Not Run
# ggplot2::ggsave(paste0(root.path, "img/", "isochrones-barriere-Italy-EN-inter-map.png"), inter.map, width = 7, height = 7)
```
<img alt="img-name" src="https://raw.githubusercontent.com/zoometh/neonet/main/doc/talks/2024-simep/img/isochrones-barriere-Italy-EN-inter-map.png"
" width="300">
<img alt="img-name" src="https://raw.githubusercontent.com/zoometh/neonet/main/doc/talks/2024-simep/img/isochrones-barriere-Italy-EN-inter-map-ex.png"
" width="300">
Weighted medians interpolated
#### EN and LM
The same function can be used symetrically: instead of plotting the earliest dates of the Neolithic (EN), one can also plot the latest dates of the Paleolithic (or Late Mesolithic, LM)
```R
myc14data <- "https://raw.githubusercontent.com/zoometh/neonet/main/results/1_AOI_France_E-W.geojson"
neo_isochr(df.c14 = myc14data, mapname = "France_EW")
neo_isochr(df.c14 = myc14data, mapname = "France_EW", selected.per = c("LM"))
```
<img alt="img-name" src="https://raw.githubusercontent.com/zoometh/neonet/main/results/France_EW-neolithic-isochr.png"
" width="700">
<img alt="img-name" src="https://raw.githubusercontent.com/zoometh/neonet/main/results/France_EW-paleolithic-isochr.png"
" width="700">
Output maps from the `1_AOI_France_E-W.geojson`: earliest Neolithic dates and latest Paleolithic dates
## Koppen
> KCC, Koppen Climate Classification, Koeppen Climate Classification
The Neonet framework and online app integrate Koppen Climate Classification (KCC) for 6,000 BP to 10,000 BP.
### Classes
Koppen classification uses five broad classes:
| code | climate | colors |
| -----|---------|--------|
| A* | Tropical | ![#0000FF](https://placehold.co/15x15/0000FF/0000FF.png) ![#0078FF](https://placehold.co/15x15/0078FF/0078FF.png) ![#46AAF](https://placehold.co/15x15/46AAF/46AAF.png) |
| B* | Arid | ![#FF0000](https://placehold.co/15x15/FF0000/FF0000.png) ![#FF9696](https://placehold.co/15x15/FF9696/FF9696.png) ![#F5A500](https://placehold.co/15x15/F5A500/F5A500.png) ![#FFDC64](https://placehold.co/15x15/FFDC64/FFDC64.png) |
| C* | Temperate | ![#FFFF00](https://placehold.co/15x15/FFFF00/FFFF00.png) ![#C8C800](https://placehold.co/15x15/C8C800/C8C800.png) ![#969600](https://placehold.co/15x15/969600/969600.png) ![#96FF96](https://placehold.co/15x15/96FF96/96FF96.png) ![#64C864](https://placehold.co/15x15/64C864/64C864.png) ![#329632](https://placehold.co/15x15/329632/329632.png) ![#C8FF50](https://placehold.co/15x15/C8FF50/C8FF50.png) ![#64FF50](https://placehold.co/15x15/64FF50/64FF50.png) ![#32C800](https://placehold.co/15x15/32C800/32C800.png) |
| D* | Cold | ![#FF00FF](https://placehold.co/15x15/FF00FF/FF00FF.png) ![#C800C8](https://placehold.co/15x15/C800C8/C800C8.png) ![#963296](https://placehold.co/15x15/963296/963296.png) ![#966496](https://placehold.co/15x15/966496/966496.png) ![#AAAF](https://placehold.co/15x15/AAAF/AAAF.png) ![#5A78DC](https://placehold.co/15x15/5A78DC/5A78DC.png) ![#4B50B4](https://placehold.co/15x15/4B50B4/4B50B4.png) ![#320087](https://placehold.co/15x15/320087/320087.png) ![#00FFFF](https://placehold.co/15x15/00FFFF/00FFFF.png) ![#37C8FF](https://placehold.co/15x15/37C8FF/37C8FF.png) ![#007D7D](https://placehold.co/15x15/007D7D/007D7D.png) ![#00465F](https://placehold.co/15x15/00465F/00465F.png) |
| E* | Polar | ![#B2B2B2](https://placehold.co/15x15/B2B2B2/B2B2B2.png) ![#666666](https://placehold.co/15x15/666666/666666.png) |
And, there are 30 basic classes
| code | num | value | hexa color | color |
|------------|-------:|:------------------------------------------|:-------|:---------------------------------------------------------|
| Af | 1 | Tropical, rainforest | 0000FF | ![#0000FF](https://placehold.co/15x15/0000FF/0000FF.png) |
| Am | 2 | Tropical, monsoon | 0078FF | ![#0078FF](https://placehold.co/15x15/0078FF/0078FF.png) |
| Aw | 3 | Tropical, savannah | 46AAF | ![#46AAF](https://placehold.co/15x15/46AAF/46AAF.png) |
| BWh | 4 | Arid, desert, hot | FF0000 | ![#FF0000](https://placehold.co/15x15/FF0000/FF0000.png) |
| BWk | 5 | Arid, desert, cold | FF9696 | ![#FF9696](https://placehold.co/15x15/FF9696/FF9696.png) |
| BSh | 6 | Arid, steppe, hot | F5A500 | ![#F5A500](https://placehold.co/15x15/F5A500/F5A500.png) |
| BSk | 7 | Arid, steppe, cold | FFDC64 | ![#FFDC64](https://placehold.co/15x15/FFDC64/FFDC64.png) |
| Csa | 8 | Temperate, dry summer, hot summer | FFFF00 | ![#FFFF00](https://placehold.co/15x15/FFFF00/FFFF00.png) |
| Csb | 9 | Temperate, dry summer, warm summer | C8C800 | ![#C8C800](https://placehold.co/15x15/C8C800/C8C800.png) |
| Csc | 10 | Temperate, dry summer, cold summer | 969600 | ![#969600](https://placehold.co/15x15/969600/969600.png) |
| Cwa | 11 | Temperate, dry winter, hot summer | 96FF96 | ![#96FF96](https://placehold.co/15x15/96FF96/96FF96.png) |
| Cwb | 12 | Temperate, dry winter, warm summer | 64C864 | ![#64C864](https://placehold.co/15x15/64C864/64C864.png) |
| Cwc | 13 | Temperate, dry winter, cold summer | 329632 | ![#329632](https://placehold.co/15x15/329632/329632.png) |
| Cfa | 14 | Temperate, no dry season, hot summer | C8FF50 | ![#C8FF50](https://placehold.co/15x15/C8FF50/C8FF50.png) |
| Cfb | 15 | Temperate, no dry season, warm summer | 64FF50 | ![#64FF50](https://placehold.co/15x15/64FF50/64FF50.png) |
| Cfc | 16 | Temperate, no dry season, cold summer | 32C800 | ![#32C800](https://placehold.co/15x15/32C800/32C800.png) |
| Dsa | 17 | Cold, dry summer, hot summer | FF00FF | ![#FF00FF](https://placehold.co/15x15/FF00FF/FF00FF.png) |
| Dsb | 18 | Cold, dry summer, warm summer | C800C8 | ![#C800C8](https://placehold.co/15x15/C800C8/C800C8.png) |
| Dsc | 19 | Cold, dry summer, cold summer | 963296 | ![#963296](https://placehold.co/15x15/963296/963296.png) |
| Dsd | 20 | Cold, dry summer, very cold winter | 966496 | ![#966496](https://placehold.co/15x15/966496/966496.png) |
| Dwa | 21 | Cold, dry winter, hot summer | AAAF | ![#AAAF](https://placehold.co/15x15/AAAF/AAAF.png) |
| Dwb | 22 | Cold, dry winter, warm summer | 5A78DC | ![#5A78DC](https://placehold.co/15x15/5A78DC/5A78DC.png) |
| Dwc | 23 | Cold, dry winter, cold summer | 4B50B4 | ![#4B50B4](https://placehold.co/15x15/4B50B4/4B50B4.png) |
| Dwd | 24 | Cold, dry winter, very cold winter | 320087 | ![#320087](https://placehold.co/15x15/320087/320087.png) |
| Dfa | 25 | Cold, no dry season, hot summer | 00FFFF | ![#00FFFF](https://placehold.co/15x15/00FFFF/00FFFF.png) |
| Dfb | 26 | Cold, no dry season, warm summer | 37C8FF | ![#37C8FF](https://placehold.co/15x15/37C8FF/37C8FF.png) |
| Dfc | 27 | Cold, no dry season, cold summer | 007D7D | ![#007D7D](https://placehold.co/15x15/007D7D/007D7D.png) |
| Dfd | 28 | Cold, no dry season, very cold winter | 00465F | ![#00465F](https://placehold.co/15x15/00465F/00465F.png) |
| ET | 29 | Polar, tundra | B2B2B2 | ![#B2B2B2](https://placehold.co/15x15/B2B2B2/B2B2B2.png) |
| EF | 30 | Polar, frost | 666666 | ![#666666](https://placehold.co/15x15/666666/666666.png) |
The Koppen Climate Classes (KCC) are listed [here](https://github.com/zoometh/neonet/blob/main/inst/extdata/koppen.tsv)
### Koppen functions
Koppen functions are designed not only for the Neonet dataset, but also for all radiocarbon dataset respecting the minimum data stracture (site, labcode, x, y, etc.).
#### Create KCC maps
The app integrates Koppen Climate Classification (KCC) for 6,000 BP to 10,000 BP created with the R [pastclim](https://github.com/EvolEcolGroup/pastclim) package and the `neo_kcc_create()` function.
<img alt="img-name" src="https://raw.githubusercontent.com/zoometh/neonet/main/doc/img/panel_map_clim_past-1.png"
" width="600">
The Koppen Climate Classification calculated for 8,000 BP (8k) with the pastclim R package and hosted on a GeoServer
Neonet functions help to blend pastclim KCC and radiocarbon dates.
```R
# install the packages pastclim and terra
devtools::install_github("EvolEcolGroup/pastclim", ref="dev")
library(pastclim)
library(terra)
# set paths and create maps
outDir <- "C:/Rprojects/neonet/doc/data/clim/"
pastclim::set_data_path(path_to_nc = outDir)
source("R/neo_kcc_create.R")
neo_kcc_create()
```
Creates these KCC GeoTiffs:
KCC are created as GeoTiffs using the R pastclim package
| Geotiff image | description |
|----------|----------|
| |koppen_11k [download](https://raw.githubusercontent.com/zoometh/neonet/main/doc/data/clim/koppen_11k.tif)|
| |koppen_10k [download](https://raw.githubusercontent.com/zoometh/neonet/main/doc/data/clim/koppen_10k.tif)|
| |koppen_9k [download](https://raw.githubusercontent.com/zoometh/neonet/main/doc/data/clim/koppen_9k.tif)|
| |koppen_8k [download](https://raw.githubusercontent.com/zoometh/neonet/main/doc/data/clim/koppen_8k.tif)|
| |koppen_7k [download](https://raw.githubusercontent.com/zoometh/neonet/main/doc/data/clim/koppen_7k.tif)|
| |koppen_6k [download](https://raw.githubusercontent.com/zoometh/neonet/main/doc/data/clim/koppen_6k.tif)|
Past Koppen Climate Classification calculated in Kyears BP with the pastclim R package
#### Extract
This set of functions creates an interactive dataset of `df.c14` dates and the climates to which they belong.
```R
kcc.file <- c("koppen_6k.tif", "koppen_7k.tif", "koppen_8k.tif",
"koppen_9k.tif", "koppen_10k.tif", "koppen_11k.tif")
source("R/neo_kcc_extract.R")
df_kcc <- neo_kcc_extract(df.c14 = df.c14, kcc.file = kcc.file)
source("R/neo_kcc_extract_longformat.R")
df_kcc_long <- neo_kcc_extract_longformat(df_kcc)
source("R/neo_dbs_info_dates_datatable.R")
dt.out <- neo_dbs_info_dates_datatable(df.c14 = df_kcc_long,
fields = c("SiteName", "code", "Period", "median", "map", "LabCode", "db_period", "db_culture", "sourcedb", "X", "Y", "color"),
font.size = "16pt")
## Not Run
# htmlwidgets::saveWidget(dt.out, "C:/Rprojects/neonet/doc/talks/2024-simep/img/dates_kcc.html")
```
#### Map
The `neo_kcc_map()` creates a KCC map with a layer of dates above
```R
source("R/config.R") # default variables: column names mapping, colors, etc.
df <- c14bazAAR::get_c14data("neonet")
df <- sf::st_as_sf(df, coords = c("lon", "lat"), crs = 4326)
neo_kcc_map(df.c14 = df,
kcc = "C:/Rprojects/neonet/doc/data/clim/koppen_7k.tif",
export = TRUE,
fileOut = "neonet_kcc.png" )
```
Gives:
<img alt="img-name" src="https://raw.githubusercontent.com/zoometh/neonet/main/doc/talks/2024-simep/img/neonet_kcc.png"
" width="700">
The neonet dataset over the KCC 7k
To assess what were the climates classes that where inhabited in the past, during the Late Mesolithic (LM) and Middle Mesolithic (MM) based on previous dates
```R
df.c14 <- neo_calib(df.c14)
df.c14 <- sf::st_as_sf(df.c14, coords = c("lon", "lat"), crs = 4326)
kcc.file <- c("koppen_6k.tif", "koppen_7k.tif", "koppen_8k.tif",
"koppen_9k.tif", "koppen_10k.tif", "koppen_11k.tif")
df_cc <- neo_kcc_extract(df.c14 = df.c14, kcc.file = kcc.file)
col.req <- gsub(pattern = ".tif", "", kcc.file)
neo_kcc_plotbar(df_cc = df_cc,
kcc.file = c("koppen_8k.tif", "koppen_9k.tif"),
col.req = col.req,
selected.per = c("EN"),
title = "Neolithic: transition btw 7,000 and 6,000 BC",
legend.show = FALSE)
```
Gives:
<img alt="img-name" src="https://raw.githubusercontent.com/zoometh/neonet/main/doc/talks/2024-simep/img/EN_kcc_stacked.png"
" width="600">
KCC occupied during the EN between 7,000 and 6,000 BC (9 ka and 8 ka BP) with counts of sites belonging to these time slices
The `neo_kcc_extract()` function collects the KCC values (climates) of each date.
## NeoNet strati
NeoNet-strati is an online R Shiny interactive app to record the stratigraphy of NeoNet's archaeological sites in an editable dataframe based on `LabCode` identifiers.
NeoNet-strati app
```mermaid
flowchart TD
A[NeoNet dataset] --is read by--> B:::neonetshiny;
B --edit
site stratigraphy--> B;
B --export
site stratigraphy
file--> C[Roc du Dourgne_2023-07-30.csv];
C --is read by--> D:::neonetfunct;
D --export --> E[maps
charts
listings
...];
classDef neonetfunct fill:#e3c071;
classDef neonetshiny fill:#71e37c;
```
Strati app and strati analysis overall Workflow
### App interface
The app is composed by different panels: a site to be recorded ([Site Stratigraphy panel](https://github.com/zoometh/neonet?tab=readme-ov-file#site-stratigraphy-panel)), and the complete dataset ([All sites panel](https://github.com/zoometh/neonet?tab=readme-ov-file#all-sites-panel)). A site name is copied from [All sites panel](https://github.com/zoometh/neonet?tab=readme-ov-file#all-sites-panel) to [Site Stratigraphy panel](https://github.com/zoometh/neonet?tab=readme-ov-file#site-stratigraphy-panel).
#### Site Stratigraphy panel
Plot a selected site in an editable table to record its stratigraphical relationships.
<img alt="img-name" src="https://raw.githubusercontent.com/zoometh/neonet/main/doc/img/app-strati-site.png"
" width="700">
Panel "Site Stratigraphy" editable dataframe. By default the app opens on "Pokrovnik"
#### All sites panel
Show the complete NeoNet dataset. A site can be selected by searching it in the selection search bar (top-right) and copying its name (`Site Name` column). Here Roc du Dourgne, highlighted in blue.
<img alt="img-name" src="https://raw.githubusercontent.com/zoometh/neonet/main/doc/img/app-strati-allsite-dourgne.png"
" width="1100">
Panel "All sites". Selection of the "Roc du Dourgne" site
<img alt="img-name" src="https://raw.githubusercontent.com/zoometh/neonet/main/doc/img/app-strati-site-dourgne-1.png"
" width="700">
"Roc du Dourgne" site sorted on its "PhaseCode"
The stratigraphical relations can be added into the "After" column, and thereafter exported in CSV
<img alt="img-name" src="https://raw.githubusercontent.com/zoometh/neonet/main/doc/img/app-strati-site-dourgne-after.png"
" width="1100">
"Roc du Dourgne" stratgraphical relationships (column "After") after edition
For example, "Roc du Dourgne" relationships are:
|LabCode |After |Period |PhaseCode | C14Age| C14SD|
|:-------|:-------|:------|:---------|------:|-----:|
|MC-1101 |MC-1102 |EN |C5 | 5050| 100|
|MC-1102 |MC-1103 |EN |C5 | 6170| 100|
|MC-1103 |MC-1105 |EN |C6 | 5100| 80|
|MC-1104 |MC-1105 |EN |C6 | 6470| 100|
|MC-1105 |MC-1107 |EN |C6 | 5550| 80|
|MC-1107 | |LM |C7 | 6850| 100|
|MC-781 | |EN |C6 | 5000| 170|
|MC-782 | |LM |Layer 7 | 5770| 170|
The first row can be read as: "*the layer containing radiocarbon date MC-1101 comes **after** the layer containing radiocarbon date MC-1102*".
Once the site stratigraphy recorded, save the changes by downloading the dataset pressing the button (top-left) as a CSV file. The output file name is the site name and the current date (ex: `Roc du Dourgne_2024-04-12.csv`).
### Strati analysis
The file exported from the NeoNet strati app can be read by the `neo_strati()` function (see [Harris Matrix](https://github.com/zoometh/neonet#harris-matrix))
#### Harris Matrix
The output CSV file exported by [NeoNet-strati](https://github.com/zoometh/neonet#neonet-strati) can be read by the `neo_strat()` function. For example, ploting the `C14Age` and the `PhaseCode`.
```R
neo_strat(inData = 'https://raw.githubusercontent.com/historical-time/data-samples/main/neonet/Roc du Dourgne_2023-07-30.csv',
outLabel = c("C14Age"))
neo_strat(inData = 'https://raw.githubusercontent.com/historical-time/data-samples/main/neonet/Roc du Dourgne_2023-07-30.csv',
outLabel = c("PhaseCode"))
```
Gives:
<img alt="img-name" src="https://raw.githubusercontent.com/zoometh/neonet/main/doc/img/app-strati-diag-dourgne-c14age.png"
" width="400">
<img alt="img-name" src="https://raw.githubusercontent.com/zoometh/neonet/main/doc/img/app-strati-diag-dourgne-phasecode.png"
" width="400">
"Roc du Dourgne" stratgraphical relationships using LabCode identifiers, ordered on the "LabCode" column, displaying the C14Age (left) and the LabCode (right)
Changing the `outLabel` to `Period` allows to color on periods using the default period colors (see the [web document](https://zoometh.github.io/neonet/doc/#3_data_panel))
```R
neo_strat(inData = 'https://raw.githubusercontent.com/historical-time/data-samples/main/neonet/Roc du Dourgne_2023-07-30.csv',
outLabel = c("Period"))
```
Gives:
<img alt="img-name" src="https://raw.githubusercontent.com/zoometh/neonet/main/doc/img/app-strati-diag-dourgne-period.png"
" width="500">
"Roc du Dourgne" stratgraphical relationships using LabCode identifiers, ordered on the "LabCode" column
## Leafrog alignment
Using `neo_leapfrog(DT = T)` to merge dataframe from NeoNet and Leapfrog on common C14 LabCode values: <https://historical-time.github.io/caa23/neonet/results/NN_and_LF.html>
Screen capture of [NN_and_LF.html](https://historical-time.github.io/caa23/neonet/results/NN_and_LF.html)
## Sibling projects
There are mainy IT-based projects dealing with radiocarbon dates, such as the comprehensive platform [Xronos](https://xronos.ch/), the radiocarbon aggregator [C14bazAAR](https://github.com/ropensci/c14bazAAR), and many others, see the [open-archaeo](https://open-archaeo.info/tags/radiocarbon-dating-calibration-and-sequencing/) site
## Documentation
* NeoNet app [web document](https://zoometh.github.io/neonet/doc/)
* [Contribution rules](https://github.com/zoometh/neonet/blob/master/github/CONTRIBUTING.md)
* [NeoNet package license](https://github.com/zoometh/neonet/blob/master/LICENSE)
## Communication
* Big Historical Data Conference
- Shiny server (with the app embeded): http://shinyserver.cfs.unipi.it:3838/neonet/bhdc
- GitHub (without the app embeded): https://zoometh.github.io/neonet/doc/talks/2023-bhdc
* Neonet workshop
- GitHub: https://zoometh.github.io/neonet/doc/talks/2024-neonet-ws/pres
* Oxford Seminar Series
- GitHub: https://zoometh.github.io/neonet/doc/talks/2024-oxford/pres
* SIMEP
- GitHub: https://zoometh.github.io/neonet/doc/talks/2024-simep/pres
* ASD-CSIC
- GitHub: https://zoometh.github.io/neonet/doc/talks/2024-asd/pres