Bulk import¶
Strategies¶
There are several ways one could accomodate bulk imports:
- offline by a sysadmin
- online by office users
- online by national coordinators
At the moment, only the first one is implemented.
We are ready to implement the next steps if there is enough demand for it.
Importing¶
Here is the outline of the proces:
- A national coordinator downloads the template file (Excel) for entering the contributions.
- They change
CC
into the ISO language code of their country,YYYY
to the relevant year, andcreator@dariah.eu.xslx
to the email address of the DARIAH user that is responsible for managing these contributions in the system. - They enter contributions in the
contributions
sheet. The instructions to do this properly are on theREADME
sheet, and where there are constrained values, the lists of those values are in other sheets in the same workbook. - They send the sheets to an office user, who send the sheets to a root user.
- The root user performs a bulk import by carrying out the steps specified below.
Overview of imported contributions¶
Once the sheet has been imported, the new contributions can be found easily via the overview page. There is a button to see only the bulk imported contributions, and by grouping them by year and country it is easy to identify the individual imports.
Contributions now have a field import
in the provenance section which shows the file name of the Excel sheet by which they have been imported. The field isPristine
shows whether these contributions have been modified since importing.
Performing an offline bulkimport¶
These steps must be taken by somebody with access to the production machine.
Start on your local machine.
- Take care that the build script is working and can be called from the terminal using the command
dab
(i.e. dariah-build). - Inspect the build script source code and look for the variable
BULK_DEV
. Make sure that this directory exists, and create a directorytodo
inside it. - Put all Excel sheets that need bulk importing into that
todo
directory.
Testing locally
You can test the import against your local development installation of the tool. Make sure you have a fairly up-to-date copy of the production database in your local dariah_dev
database.
Give the command
dab bulk d i
Inspect the output of the command: when things go wrong it will be clearly indicated.
When things go right, the Excel sheet will be moved from todo
to a new done
folder.
When things do not go right, make the necessary changes to the spreadsheets and run the command again.
The bulk import will not create duplicate imports, you can repeat the command as often as you like.
If you want to start from the beginning, and remove all imported contributions, make sure the Excel sheets are back in todo
again and say:
dab bulk d x
If you are ready to import in production, make sure that the Excel sheets are in todo
.
Still on your local machine:
-
Give the following command:
dab databulk p
This will copy the sheets to the production machine.
Move over to the production machine.
-
Give the command
dab bulk i
This will stop the webserver, bulk import the Excel sheet that has been copied over in the previous step, and restart the webserver again.