Labs Participation Reports
Legacy
| Area | reporting |
| Language | Python |
| Description | An experimental version of Participation Reports with additional data |
| Production URLs | |
| Documentation | [http://gitlab.com/crossref/labs/crossref-labs-reports] |
| Quality | No Sentry, no SONAR |
| Source Code | |
| Products |
The Labs Participation Reports system amalgamates quarterly billing data alongside other data points (e.g. billing data, TLDs) that are used actively by the Support team. Without its functionality the support team would have many more billing reports to run every quarter and many more support tickets. This product is, therefore, depended upon by support and a quasi-production service.
Infrastructure/Hosting
There are two components for Labs Participation Reports:
- The labs-participation-reports repository;
- and the billy-bob report generation system/repository
Labs-participation-reports
This main display application is written in Python/Streamlit and runs on fly.io with a domain mapping from prep.labs.crossref.org.
Billy-Bob
This script, which generates billing data for the main application, is written in Python, and is required to run locally. It requires a VPN or SOCKS5 connection to the data center and a user account with “root” permissions. Tim Pickard is, currently, the person who can provide setup details on the former of these (the VPN), while Isaac Farley was able to enable an account to have “root” permissions.
Command sequence
Activate your VPN or SOCKS5 proxy to the data center, then:
- Download_latest_member_list.sh
- get_member_billing_metadata.py –verbose –members-file members.json –results-dir billing_metadata
- generate_billing_files.py –verbose –quarter Q2 –results-dir billing_metadata
This will take several hours.
Once complete, from the Labs Participation Reports directory, run:
- ./update_quarterly_data.py –src-dir=billybob/billing_metadata –dst-dir=data
Fly.io / Loco
The site itself is hosted on fly.io with a virtual host setup on the “loco” Hetzner robot machine (148.251.10.165).
The relevant files are those with “prep” in their name in /etc/apache2/sites-enabled/. These files invoke the Let’s Encrypt certbot to ensure an SSL certificate is used.
These files also contain the lines:
ProxyPass / "http://prep.fly.dev:80/
ProxyPassReverse / "http://prep.fly.dev:80/
Which setup the reverse proxy.
Eventually, this virtual host mechanism will be moved to AWS. It may be worth considering whether we should move the Fly.io container also to AWS at that point.
