Troubleshooting: off-site archive and scheduled jobs
Core module 3PL add-on For: Administrator, Compliance manager, Warehouse staff Checked on 18.0.0.2.0, 18.0.1.0.0
What to do when the off-site S3 archive refuses its settings or its uploads fail, when a scheduled job doesn't do its work, or when a reminder goes to the wrong person. Entries about uploads first, then settings, then jobs and reminders. The Last Error texts below are shown on each archive job (Inventory ‣ Rx Tracking ‣ S3 Archive Queue, then open the job).
Last Error: "No reply from S3 (…)"
What you see: on a pending or failed archive job:
No reply from S3 (ERROR-TYPE): DETAIL
For example No reply from S3 (ConnectionError): HTTPSConnectionPool(host='storage.example.com', port=443): Max retries exceeded …, where
the host is the one your archive settings name.
When: each time the job DSCSA: send S3 archive queue tries the upload, in Fix and retry failed archive uploads.
Cause: the Odoo server got no answer from S3: no network, a firewall, a wrong Endpoint URL, a timeout, or S3 was unavailable. The upload may still have arrived; the next attempt finds out.
Remedy: resolution.
- Check that the Odoo server can open HTTPS connections to S3, and that Endpoint URL in the archive settings is empty for AWS (or the right address for S3-compatible storage). See Set up the off-site S3 archive.
- Wait for the automatic retries, or select the jobs and Retry.
Who can fix it: the network and the settings: an administrator with Administration: Settings who is also Inventory Administrator and Rx Tracking Manager; Retry: Rx Tracking Manager.
Last Error: "The S3 archive settings of … are incomplete: …"
What you see:
The S3 archive settings of COMPANY are incomplete: MISSING-SETTINGS.
MISSING-SETTINGS names what is empty, from "bucket, region, access key ID, secret access key", for example "secret access key".
When: when the job DSCSA: send S3 archive queue tries the upload. Each such try counts as an attempt.
Cause: the archive is on, but a setting the upload needs is empty: most often no secret key was ever saved (the settings show "No secret key is set.").
Remedy: resolution. Fill in the missing settings and save (Set up the off-site S3 archive), then Retry the jobs (ARC-06).
Who can fix it: an administrator with Administration: Settings who is also Inventory Administrator and Rx Tracking Manager.
Last Error: "HTTP 403 Forbidden: SignatureDoesNotMatch …" or another "HTTP …" error
What you see:
HTTP STATUS REASON: S3-ERROR-CODE - S3-ERROR-MESSAGE
For example HTTP 403 Forbidden: SignatureDoesNotMatch - ….
When: when the job DSCSA: send S3 archive queue sends the upload and S3 refuses it.
Cause: S3 answered with an error. The code after the status is S3's own: SignatureDoesNotMatch is a wrong secret access key;
InvalidAccessKeyId a wrong access key ID; AccessDenied an IAM policy that doesn't allow the upload; NoSuchBucket a wrong bucket
or region. The secret key never appears in the message.
Remedy: resolution. Correct the setting in the archive settings, or the IAM policy or bucket with your AWS administrator; then Retry the jobs (ARC-06).
Who can fix it: an administrator with Administration: Settings who is also Inventory Administrator and Rx Tracking Manager, with the AWS administrator.
Last Error: "S3 already holds an object under this key, and this job never stored one …"
What you see: on a Failed job:
S3 already holds an object under this key, and this job never stored one: another database or tool writes to the same prefix. The object
was not replaced. Check the bucket, and use a key prefix of its own for this database.
When: when the job DSCSA: send S3 archive queue sends the upload. The job is Failed at once, without more attempts.
Cause: an object with the same key is already in the bucket, written by something else: typically a second Odoo database (a copy, or another company database) with the same Key prefix. The archive never replaces an object.
Remedy: resolution.
- Find out which database or tool wrote the object, and stop it writing to this prefix.
- Give each database its own Key prefix in the archive settings (Set up the off-site S3 archive).
- Retrying this job gives the same result, because its key is unchanged. Ask your implementer how to archive the record under the new prefix.
Who can fix it: an administrator with Administration: Settings who is also Inventory Administrator and Rx Tracking Manager, with the AWS administrator.
A job that is Done with the note "S3 already holds this key: an earlier attempt, whose reply was lost, stored it. Version ID and ETag are not known; check the object in the bucket if needed." is fine: an earlier attempt stored the object but its reply was lost.
Last Error: "The content of … no longer matches the SHA-256 recorded when it was queued …"
What you see: on a Failed job:
The content of S3-KEY no longer matches the SHA-256 recorded when it was queued. It was not sent. Run Verify Integrity on the DSCSA
documents.
When: when the job DSCSA: send S3 archive queue prepares the upload. The job is Failed at once.
Cause: the stored file changed after it was queued. The archive refuses to send a file that differs from the one recorded when the document was posted.
Remedy: resolution. Run Verify Integrity on the documents and follow what it reports (Verify the integrity of the DSCSA documents). Don't retry the job until the cause is known.
Who can fix it: Rx Tracking Manager.
Last Error: "Unexpected error: …" or "The access key ID contains characters that are not allowed."
What you see: Unexpected error: ERROR-TYPE, or The access key ID contains characters that are not allowed.
When: when the job DSCSA: send S3 archive queue tries the upload.
Cause: "The access key ID contains characters …": the Access key ID in the archive settings has a space, a line break or another character an access key can't have, often from copying. "Unexpected error": something else went wrong in Odoo; the server log has the details.
Remedy: resolution. Enter the Access key ID again, exactly as AWS shows it, and save; then Retry. For "Unexpected error", give the job's key and the time to whoever runs your Odoo server.
Who can fix it: an administrator with Administration: Settings who is also Inventory Administrator and Rx Tracking Manager.
Archive jobs stay Pending
What you see: jobs in Inventory ‣ Rx Tracking ‣ S3 Archive Queue stay Pending with Attempts 0, or their Next
Attempt has passed and nothing happens.
When: after documents are posted or retention exports made, in Watch the archive queue and each record's archive state.
Cause: one of these:
- the job DSCSA: send S3 archive queue doesn't run (the server runs no scheduled jobs, or the job was switched off);
- the archive is off for the company: its jobs wait until it is on again;
- the company is in test mode: compliance jobs wait (only Governance (test) jobs are sent); or it left test mode: its unsent test jobs are Superseded, not pending.
Remedy: resolution.
- Check the job and run it by hand: Check that a DSCSA job ran, and run it by hand.
- Check the archive settings: Off-site write-once archive (S3) selected, and Test mode as you want it.
Who can fix it: Administrator (Administration: Settings).
Error: "Oops! Something went wrong" on Queue Unarchived Records
What you see: after selecting Queue Unarchived Records in the S3 Archive Queue, the dialog:
Oops!
Something went wrong... If you really are stuck, share the report with your friendly support service
Its technical details end with TypeError: DscsaArchiveJob.action_dscsa_queue_unarchived() takes 1 positional argument but 2 were given.
When: in Queue records posted before the archive was switched on, and after switching test mode off (ARC-02).
Cause: a product defect (PF-W11-01): the button can't start the action it belongs to. Nothing is queued.
Remedy: workaround. Create the Queue unarchived DSCSA records (S3) action once and run it from the documents list: parts 1 and 2 of Queue records posted before the archive was switched on.
Who can fix it: part 1: Administrator (Administration: Settings); part 2: Rx Tracking Manager.
Error: "Enable the off-site S3 archive in the Inventory settings first."
What you see: Enable the off-site S3 archive in the Inventory settings first.
When: when you queue records posted before the archive was switched on (ARC-07).
Cause: none of the companies selected in the company switcher has the archive on.
Remedy: resolution. Select the company whose records you want to queue in the company switcher, or switch its archive on (ARC-01); then queue again.
Who can fix it: Rx Tracking Manager (company switcher); the settings: an administrator with Administration: Settings who is also Inventory Administrator and Rx Tracking Manager.
Error: "Only failed or pending archive jobs can be retried."
What you see: an Invalid Operation dialog: Only failed or pending archive jobs can be retried.
When: when you select Retry in Fix and retry failed archive uploads.
Cause: every selected job is Done or Superseded. Those are never sent again.
Remedy: resolution. Filter the queue on Failed (or Pending), select those jobs, and select Retry again.
Who can fix it: Rx Tracking Manager.
Error: "Only DSCSA managers can manage the S3 archive queue."
What you see: Only DSCSA managers can manage the S3 archive queue., or no S3 Archive Queue menu item and no Retry or Queue
Unarchived Records button.
When: when a user who isn't an Rx Tracking Manager tries to open or work the queue.
Cause: only Rx Tracking Managers manage the queue. Rx Tracking Users can read a document's archive jobs on its Off-site Archive tab.
Remedy: resolution. Ask an Rx Tracking Manager to retry or queue, or ask an administrator for the Manager right. See Error: "Only DSCSA managers can …".
Who can fix it: Administrator (Administration: Settings), for the right.
Error: "S3 archive jobs are queued only by the archive itself." and related refusals
What you see: one of:
S3 archive jobs are queued only by the archive itself.
S3 archive jobs are written only by the archive queue (refused: FIELDS).
The S3 archive references (FIELDS) are written only by the archive queue.
When: when someone imports, creates or edits archive jobs, or tries to change a document's archive fields (S3 Archive, S3 Manifest Key …), for example through an import or an integration.
Cause: only the archive writes its jobs and the archive references, so the record of what is in the bucket can't be changed by hand.
Remedy: none needed: nothing is changed. To send a job again, use Retry (ARC-06).
Who can fix it: nobody; this is intended.
Error: "Invalid fields: S3 Bucket, S3 Access Key ID"
What you see: a notification when you save the settings:
Invalid fields:
S3 Bucket
S3 Access Key ID
Bucket and Access key ID are outlined in red. (The notification uses the fields' technical labels, PF-W11-03.)
When: when you select Save in Set up the off-site S3 archive with Off-site write-once archive (S3) selected.
Cause: Bucket, Region and Access key ID are required while the archive is on. The notification lists the empty ones.
Remedy: resolution. Fill them in and save again, or clear Off-site write-once archive (S3).
Who can fix it: an administrator with Administration: Settings who is also Inventory Administrator and Rx Tracking Manager.
Error: "Invalid S3 bucket name …"
What you see: a Validation Error dialog:
Invalid S3 bucket name "BUCKET": use 3 to 63 lower-case letters, digits, "-" or ".", starting and ending with a letter or digit.
When: when you save the archive settings (ARC-01).
Cause: the Bucket isn't a valid S3 bucket name, for example Bad_Bucket (capitals and _ aren't allowed).
Remedy: resolution. Enter the bucket's name exactly as it is in AWS, and save.
Who can fix it: an administrator with Administration: Settings who is also Inventory Administrator and Rx Tracking Manager.
Error: "The region may only contain lower-case letters, digits and "-" …"
What you see: a Validation Error dialog:
The region may only contain lower-case letters, digits and "-", e.g. "us-east-1".
When: when you save the archive settings (ARC-01).
Cause: the Region has spaces or capitals, for example US East.
Remedy: resolution. Enter the region code as AWS writes it, for example us-east-1 or us-west-2, and save.
Who can fix it: an administrator with Administration: Settings who is also Inventory Administrator and Rx Tracking Manager.
Errors about the endpoint URL
What you see: a Validation Error dialog with one of:
The request URL must start with http:// or https://.
The request URL must have a host name and no user name or password.
The request URL has an invalid port.
The endpoint URL must not have a query string or #fragment.
When: when you save the archive settings (ARC-01) with an Endpoint URL.
Cause: the Endpoint URL isn't a plain web address: another scheme (ftp://…), a user name or password in it (https://user:pw@…),
a port above 65535, or a ?… or #… part.
Remedy: resolution. For AWS, leave Endpoint URL empty. For S3-compatible storage, enter only the scheme, the host and, if needed,
the port, for example https://storage.example.com.
Who can fix it: an administrator with Administration: Settings who is also Inventory Administrator and Rx Tracking Manager.
Error: "The S3 key prefix may only contain letters, digits …"
What you see: a Validation Error dialog:
The S3 key prefix may only contain letters, digits, ".", "_", "-" and "/" between folder names, e.g. "dscsa" or "archive/dscsa".
When: when you save the archive settings (ARC-01).
Cause: the Key prefix starts or ends with /, has two / in a row, a space or another character, or a folder named . or ...
Remedy: resolution. Enter folder names separated by single /, for example dscsa or archive/dscsa, and save.
Who can fix it: an administrator with Administration: Settings who is also Inventory Administrator and Rx Tracking Manager.
Error: "The S3 archive retention must be between 6 years (the DSCSA minimum) and 30 years."
What you see: a Validation Error dialog: The S3 archive retention must be between 6 years (the DSCSA minimum) and 30 years.
When: when you save the archive settings (ARC-01).
Cause: Retention (years) is below 6 or above 30. The upper limit catches typos (60 for 6), because a compliance lock can never be shortened.
Remedy: resolution. Enter a number from 6 to 30, as your procedure decides, and save.
Who can fix it: an administrator with Administration: Settings who is also Inventory Administrator and Rx Tracking Manager.
Error: "The S3 archive test retention must be between 1 and 30 days."
What you see: a Validation Error dialog: The S3 archive test retention must be between 1 and 30 days.
When: when you save the archive settings with Test mode selected (ARC-02).
Cause: Test retention (days) is 0, negative or above 30.
Remedy: resolution. Enter a number from 1 to 30, and save.
Who can fix it: an administrator with Administration: Settings who is also Inventory Administrator and Rx Tracking Manager.
The archive settings aren't shown
What you see: the DSCSA (Rx Tracking) block in Inventory ‣ Configuration ‣ Settings has no Off-site write-once archive (S3) setting; or the Settings app shows no Inventory settings at all; or opening the settings answers:
You are not allowed to access 'Config Settings' (res.config.settings) records.
When: when you open the settings in Set up the off-site S3 archive.
Cause: the archive settings need three rights together: Administration: Settings (else the access error), Inventory Administrator (else no Inventory settings) and Rx Tracking Manager (else no archive setting in the block).
Remedy: resolution. Ask an administrator to give you the missing right, or to make the change.
Who can fix it: Administrator (Administration: Settings).
Error: "Only DSCSA managers with access to the Settings can set the S3 archive secret key."
What you see: Only DSCSA managers with access to the Settings can set the S3 archive secret key.
When: when a user who isn't both an Rx Tracking Manager and a Settings administrator tries to store the secret key, for example through an import or an integration.
Cause: only DSCSA managers with Administration: Settings may set the key.
Remedy: resolution. Ask an administrator who has both rights to enter the key in the archive settings (ARC-01).
Who can fix it: an administrator with Administration: Settings who is also Inventory Administrator and Rx Tracking Manager.
A copy of the database still has the archive on
What you see: in a staging or restored copy, Off-site write-once archive (S3) is still selected and the settings read "A secret key is set.".
When: in Keep a staging or restored copy from writing to the archive.
Cause: the copy was not neutralized when it was made (a plain restore). It can write into the production bucket, where every object stays locked for years.
Remedy: resolution. At once, clear Off-site write-once archive (S3) in the copy's settings and select Save, or have the server
administrator run odoo-bin neutralize -d COPY-DATABASE, which also deletes the secret key.
Who can fix it: an administrator with Administration: Settings who is also Inventory Administrator and Rx Tracking Manager; the server administrator for neutralization.
Error: "Oops! Something went wrong" when the license reminder job runs
What you see: after Run Manually on DSCSA: license expiry reminders, the dialog "Oops! Something went wrong…". Its technical
details mention ValueError("invalid literal for int() with base 10: 'VALUE'"). When the job runs on schedule, nothing is shown, and no
license gets a reminder.
When: in Check that a DSCSA job ran, and run it by hand.
Cause: the system parameter aglow_rx_tracking.license_expiry_reminder_days isn't a whole number, for example 30 days (known issue
PF-A01-07).
Remedy: resolution. Correct the value to digits only, or delete the parameter to go back to 30 days (Tune the jobs through system parameters); then run the job by hand.
Who can fix it: Administrator (Administration: Settings).
A reminder went to the wrong person, or to nobody
What you see: a DSCSA activity is assigned to someone other than expected, to OdooBot, or no activity appears.
When: for the reminders in Work the DSCSA activities in your to-do list.
Cause: one of these:
- License expiring: the license's Responsible comes first, even when the activity type has a Default User. Licenses created by data files, the demo data or scripts have OdooBot as Responsible (PF-A01-08), so their reminders go to OdooBot. A license with no Responsible and a type without Default User gets no reminder. The license is outside the reminder window, or its reminder for that expiry date was already given (one per expiry date).
- GLN missing: the contact already had an open GLN missing activity, so no second one was made; without a Default User it goes to the company's first Rx Tracking Manager.
- Trace request due: the Responsible or Received On was changed after the request was saved (PF-A04-03).
- The job that makes the reminder didn't run.
Remedy: resolution.
- Set the license's Responsible to a person (Act on license-expiry reminders), or choose a Default User (Choose who receives the DSCSA reminder activities).
- To move an activity that already exists, select Edit on it in the record's chatter and change the person.
- Check that the job ran (AUTO-01).
Who can fix it: Rx Tracking Manager for licenses and activities; Administrator (Administration: Settings) for the activity types and jobs.
For emails that never arrived and jobs that never run at all, see An email or a reminder never arrived and A scheduled job didn't run.