
Process
Below is a detailed list of steps followed to create and provide ongoing updates to intertextual maps from a geo-encoded text.
- ONE TIME/ONGOING: Encode text in TEI/XML. Places designated with the following elements and attributes:
<rs type=“place” ref=“PLACECODE”
>
</rs
>
<placeName ref=“PLACECODE”></placeName>
<rs type=“demonym” ref=“PLACECODE”></rs>
- ONE TIME: Upload file to Exist-DB corpus collection
- ONE TIME: Create blank CODEplaces.xml file
- NIGHTLY (cronjob): Export place data from tei file: create-CODE-places.xql—>CODEplaces.xml
- ONE TIME: Create xsl file to update csv file: csv-transform-obraCODE.xsl
- ONE TIME: Create csv file with stylesheet: places-obraCODE.xsl-->places-obraCODE.csv
- ONE TIME: Add csv file to savecsvfiletopublicfolder.xq
- NIGHTLY (cronjob): Hit xml/CODEplaces.xml to update csv file
- NIGHTLY (cronjob): Hit xql/savecsvfiletopublicfolder.xq
- NIGHTLY (cronjob): Import data into mysql db
- NIGHTLY (cronjob): Export mysql data (mysql table dump)
- NIGHTLY (cronjob): Create placeography file: CREATE-CODE-PLACEOGRAPHY.xql—>CODE-placeography.xml
- NIGHTLY (cronjob): Set xml file to update daily:
wget xql/create-CODE-places.xql >/dev/null 2>&1
- NIGHTLY (cronjob): Set csv file to update
wget xml/CODEplaces.xml >/dev/null 2>&1
- NIGHTLY (cronjob): Import csv into database:
mysql --execute="CREATE TABLE placeobras_import (placeref VARCHAR(150) NOT NULL,placeworks VARCHAR(250) NOT NULL, PRIMARY KEY (placeref));"
mysql --execute="LOAD DATA LOCAL INFILE 'places-obraCODE.csv' INTO TABLE placeobras_import FIELDS TERMINATED BY ',' lines terminated by '\n' IGNORE 1 LINES (placeref, placeworks)"
mysql --execute="UPDATE placeobras_import B INNER JOIN places A USING (placeref) SET A.placeworks = CASE WHEN A.placeworks ='' THEN B.placeworks WHEN A.placeworks LIKE concat('\%',B.placeworks,'\%') THEN A.placeworks ELSE concat(A.placeworks,',',B.placeworks) END;"
mysql --execute="load data local infile 'places-obraCODE.csv' IGNORE into table places fields terminated by ',' OPTIONALLY ENCLOSED BY '\"' lines terminated by '\n' IGNORE 1 LINES (placeref,placeworks)"
mysql --execute="DROP TABLE placeobras_import;"
- ONE TIME: Data entry for each place in MySql DB (using Sequel Pro)
:
geo coords: point, area or path (WKT format)
place name
place type
placewikiid
place country
placeapprox
Country/province/region shapes: https://www.gadm.org/download_country_v3.html hand-drawn: QGIS and https://arthur-e.github.io/Wicket/sandbox-gmaps3.html
- ONE TIME: Create views in MySQL DB
places_points_CODE places_areas_CODE
CREATE VIEW places_areas_CODE AS
SELECT placeref, placename,placearea,placetype,placeworks,placeapprx,placecountry
FROM places
WHERE placepointcoords<>'' and placeworks LIKE '%OBRACODE%'
- NIGHTLY (cronjob): Remove old shape files and create new shape files for place type contains OBRACODE results:
rm /srv/shiny-server/CODE/shapefiles/*
ogr2ogr /shapefiles MYSQL:DATABASE,user=USER,password=PASSWORD,port=3306 places_points_CODE places_areas_CODE -lco ENCODING=UTF-8
- ONE TIME: Create maps in R/Shiny
. Load point and area data (global)
—map data (server)
—create layout (ui) (upload to server)
- ONE TIME: Create CODE-placeography.xml file
- NIGHTLY (cronjob): Populate CODE-placeography.xml file with CREATE-CODE-PLACEOGRAPHY.xql file
- ONE TIME: Stylesheet to format CODE-placeography.xml: xsl/placeography-html-CODE.xml
- ONE TIME: Create snippet file: PLACEOGRAPHY-BIBLIO-CODE.xml
- ONE TIME: Test placeography file, e.g.: http://45.56.98.26:8080/exist/rest/db/madrid/xml/dqm-placeography.xml
- NIGHTLY (cronjob): Get snippets: xql/GETTING-SNIPPETS-PLACES-CODE.xql. Check book structure (parte/apartado/párrafo) and adjust code in get snippets and CODE-placeography.xml
- ONE TIME: Check map. Look for misplaced points and regions. Check that updates are working nightly.
Presentations:
|