MAGIC datacenter datatransfer

More information

This site shows the transfer of the MAGIC data files from the observatory at La Palma to the data center at the Port d'Informació Científica (PIC), in Barcelona, file by file and in real time.

Browsing the transfers

The files are grouped by data type (e.g. Star Data M1) and by observation night. The status of a night or a data type is the least advanced status of its files: a night shows an error as long as one of its files has one, so problems can be followed down to the file.

  • Home: the data types, to browse their nights and files.
  • Date: the transfers of all the data types of one night.
  • Activity: the latest status changes.

The page of each file shows its status history and its metadata: its paths at La Palma and at PIC, its checksum and its transfer job.

The transfer workflow

Several agents move each file through these steps:

  1. At La Palma, new files are added to the transfer queue, and their checksum (adler32) is computed.
  2. At PIC, the files are transferred with FTS, the File Transfer Service, and the jobs are followed until they end. Transfers that fail for a temporary reason are retried.
  3. At PIC, each transferred file is verified: its checksum must be the same as at La Palma.
  4. At La Palma, the observatory's database records that the file is safe at PIC.

Support

This site is owned and maintained by the Port d'Informació Científica (PIC). For any question, suggestion or problem, write to magic.support@pic.es.

Status of the files

Submitted File added to the transfer queue
To transfer Transfer ready to start
Transferring Transfer ongoing
In PIC Transfer finished, pending to verify the file at PIC
Registered Transfer finished, pending to update status in LPdb
Done Transfer workflow finished
Error: Not found Transfer not started, could not find the data file in LP
Error: Checksum Transfer not started, error in computing the checksum at LP
Error: Transfer Transfer failed
Error: Register Transfer finished, error in verifying the file at PIC
Error: Markdone Transfer finished, error in updating status in LPdb