| Related services | |
| Tags |
Cayenne ingestion
graph LR;
allmembers(All Members API);
permember(Per-member API);
cayenne/task/ingest-members(REST API Task
Ingest Members);
cayenne/api/members(REST API
/v1/members);
cayenne/es/members(REST API
ES Members Table);
allmembers-->cayenne/task/ingest-members;
permember-->cayenne/task/ingest-members;
cayenne/task/ingest-members-->cayenne/es/members;
cayenne/es/members-->cayenne/api/members;
Members are ingested from a location configured as [:upstream :prefix-info-url], currently http://doi.crossref.org/getPrefixPublisher/?prefix=.
A call to http://doi.crossref.org/getPrefixPublisher/?prefix=all is first made to obtain a full list of all members in JSON format. A member’s entry in this JSON file contains very basic information, including a list of member’s prefixes.
Then, for each prefix Cayenne calls http://doi.crossref.org/getPrefixPublisher/?prefix=<prefix> to obtain the information about the prefix in XML format, including the reference visibility. This information is added to the member’s record.
The members’ records are then indexed using Elasticsearch’s bulk API with a series of index actions.
Members are ingested once a day.
