Troubleshooting Data Destinations

 

About

After adding or updating a ProntoForms Data Destination, it is important to test it thoroughly before using it in production. 

 

Destination Executions

A Data Destination executes each time that a form linked to the data destination is submitted.  When it executes, the Data Destination then attempts to route data to the configured location.  Executions (one per destination per data record) are listed in the web administration portal for 45 days, the same period that data records are retained.  The web portal offers several ways to view the details of a data destination execution, making it easy to determine if filters, connections, and other options are configured correctly.  It also clearly lists details of execution failures.  When incorrect configuration results in a failure, it is simple to update the data destination and re-execute failed executions to try routing data again.

Problem Contact Email Address

A key tool in testing and troubleshooting Data Destinations is ensuring that your team has a Problem Contact Email Address configured. This is the easiest way to gain visibility to data destination execution failures; an email notification will be sent to the provided address. The problem contact email address can be configured through Team Settings (under General Settings).  

[ top ]

 

Testing Data Destinations

The only way to test a data destination is to submit a data record against a form linked to the data destination. While some data destination configuration issues will become obvious when a form is submitted (like if the execution fails), other configuration errors may not be immediately noticeable.  Without thorough testing, these errors can make it into production before being spotted.  Save yourself production errors and delays by testing your destinations before deploying to your users.

Connection

A connection that is not functional will cause data destination executions to fail.  Submit a form linked to the data destination, check that the execution has not failed, and check that every document/notification was successfully delivered to the configured locations/recipients.

Activate/Deactivate Data Destinations

Data destinations must be activated to receive submitted data.  In the Destination Basics tab, toggle Activate destination to receive data records at this location to enable or disable the data destination. 

Note:  Deactivated data destinations will still appear in the Destinations tab in the form builder, but it will not receive any data submissions. 

 [ top ]

 

Filtering

Ensure that any filtering that is configured functions correctly, and the destination only executes when you want it to.  This is particularly important in the case of Custom Filter rules, which use regular expressions, and a single incorrect character may break the expression.  Test filter rules by submitting forms in a test environment, testing all answers referenced in filter rules to determine if the rules behave as expected.

Always Test:

  • Answers that should pass the filter rules
  • Answers that should NOT pass the filter rules

You will want to test other things based on your specific filter rules and the use of your form.  It is best if questions referenced in filter rules have pre-defined answers (dropdown, radiobutton, multi-select, Yes/No), as it is impossible to A) predict all potential typos/errors made by mobile users and B) build a regular expression to accommodate them.

[ top ]

 

Uses of DREL

Most fields that you can enter DREL in will show sample text underneath the field as you are configuring it, so make use of this.  For example, if %u[name] is entered in the field, the sample text should list a "sample" username -- if it doesn't, something has been mis-entered.  

Data Destinations might use DREL in a number of places:

  • Folder Expressions
  • Filename Expressions
  • Email Subject Lines
  • Pushing data to specific fields
  • And more...

Any question referenced in DREL should be marked as "Required" in the form, particularly if it is the only field in the expression; if a filename expression ends up empty, for example, the transmission of data to most destinations will fail.  Test that this cannot happen in your forms.  

[ top ]

 

Data Types

Some data destinations push data from a selected ProntoForms field to a specific field in the destination service.  Salesforce Data destinations require this, for example.   Salesforce has validation set on many of their fields -- an email field will only accept a correctly formatted email, and an integer field will only accept numbers.  If data pushed to Salesforce doesn't match the validation, no record will be created in Salesforce.  For these data destinations, and others that push to services that use validation, it is best to ensure that the ProntoForms data type matches the validation.  Be sure to test the form and make sure that users cannot enter anything into these fields that the destination service will not accept.  

[ top ]

 

admin.pngnouser.pngnomobile.png

Viewing Execution Details

Once  data records have been submitted against forms linked to the Data Destination, there are a few ways to view execution details.

1. Data Destinations Execution Statistics

Navigate to the desired Data Destination's page.  On the right side of the browser there will be a table labeled Execution Statistics, showing the number of data records that have been processed by the Data Destination. Clicking on any of the states (Active, Successful) will bring up a page with the list 

execution_statistics.jpg

 

Active:  The number of data destination executions in process. 

Successful:  The number of data destination executions that were successful.

Skipped: If a data destination has been "skipped," a data record failed to save properly and the data destination was not processed.  This can occur for a number of reasons; contact ProntoForms support for troubleshooting help.

Filtered: The number of data destination executions with submitted data that did not meet the filtering rules.

Failed: The number of data destination executions that have failed.

 [ top ]

 

2. Data Record Processing Details

To see more detailed data on each execution, navigate to the Data & Analytics tab in the main menu, and select Data.  All submitted data records will be listed; if a data destination execution failed there will be a red "X" icon beside the name.  Click on the data record reference number to view details.  

failed.jpg

Data Record Processing

The data record processing table shows details on the execution of each processing step of the data on the ProntoForms server. Each step occurs in the order that it is listed.  Note that this table does not include any information about processing/handling of the data in the mobile app, before the form was submitted.  

The standard processing steps are as follows:

  • Save Data:  Saves the data to the server.  The subsequent steps will not be completed if "Save Data" fails.   Contact our support team if this step fails.
  • Resolve Geo Address: Will be skipped if no geo location was collected by the device and attached to the form.  
  • Process Images: reading of images collected with the form.  Will be skipped if no images are attached.  If this step fails, the subsequent steps can still process.
  • Data Destination: A new row will be listed for each data destination linked to the form.  If one data destination execution fails, the others can still execute.  

Symbols: Processing Status

check.jpg - The step has been successfully processed.

cross-circle.png  - The step has failed.  See details for more information.  

SKIP.png - The step has been skipped.  This usually happens because the step is not applicable.

    • The "Process Images" step will be skipped if there are no images in the form.
    • A Data Destination execution will be skipped if the data record does not meet the destination's filter rules.

faileddetails.jpg

 [ top ]

 

Troubleshooting Data Destinations

Skipped

A Data Destination execution will only show as "skipped" if the submitted data does not meet filter rules.  Either adjust the filter rules, or change your source data so that it will meet filter rules.  Details will be provided about the string that is being filtered.  In the sample below, the question referenced in the filter rule is not returning a value, causing the email data destination to skip executing, and not send an email.

skipped.png

 

Failed

Occasionally, a data destination will fail due to a glitch or a momentary outage in either ProntoForms or the external service the destination is linked to.  Sometimes simply re-executing the failed execution (see below) will resolve the issue.  

The error causing the problem will be listed in the "details" column of the data record processing table.   The messages listed can vary widely, but are typically reasonably descriptive and make the cause quite clear. One common problem is security solutions such as firewalls blocking ProntoForms IP addresses.  If you are uncertain of the cause, feel free to contact ProntoForms Support and we will look into the details of the error for you.

 [ top ]

 

admin.pngnouser.pngnomobile.png

Re-Executing a Failed Execution

Automatic Re-Executions

If a data destination execution fails, the ProntoForms server will automatically re-attempt again until either it executes successfully, or the data record is removed from the ProntoForms system (records are kept in the ProntoForms system for up to 45 days from creation from the device or from an initial dispatch).   When a data destination execution fails, only the data record that failed to process will be re-executed.  Other data records successfully processed  by the data destination will not re-execute.

The first and second attempts at re-execution occur after 30 minutes.  The server will continue to retry in an "exponential back-off" process (increasingly longer intervals between attempts). This is done in order to avoid burdening a service with many retry transactions from the ProntoForms system.  Subsequent retries occur after 1 hour, 10 hours,  2.8 days, etc.   The retry process can be stopped by resolving the issue in the destination system, or by deleting the record in the ProntoForms system.  Just like with the initial execution attempt, each time a re-execution fails, an email notification is sent to the “Problem Contact Email” address for the team. 

 

Automatic Re-Executions and the Data Record Pass-Through Feature

If using the data record pass-through feature,  submitted data is deleted from the ProntoForms system as soon as it has successfully processed.  If there is an issue in processing, the data is held for up to 24 hours.  This means that there will only be 4 automatic re-executions (within the 24 hour period) before the data is deleted.  If an issue with your data destination is not resolved within 24 hours, data may be lost.

It is highly recommended for users of this feature to configure a redundant data destination to help safeguard against loss of data.  This data destination should ideally route data to a different location, allowing you to back up your data if there is an issue or outage with the main service you are routing your data to. 

 [ top ]

 

Manually Re-Execute a Failed Execution

A re-execution can be initiated manually through the ProntoForms web administration as well.  This can be a useful after correcting configuration issues with data destination, to test if the changes are effective (or if it is simply inconvenient to wait until the next automatic execution attempt).

Navigate to the desired data destination, and click the “Re-execute Failed Executions” button to re-process all failed data records associated with the data destination. 

reexecutebutton.jpg

 

Alternatively, enter the Data Records tab and click the “Re-Execute Failed Executions” icon as circled in red below.  Do NOT select the icon to the right of this, which will re-execute ALL executions, as this will greatly increase processing time.

reexecuteicon.jpg

 

It is also possible to re-execute the single failed step in an execution; click on the desired data record name, then mouse over the desired step.  A “re-execute” icon will appear to the right. 

reexecutestep.jpg

[ top ]

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments

  • Avatar
    Huberto Garza

    "retry in an "exponential back-off""

    epic.

  • Avatar
    Carla Messer

    Where / how do you change the email where the notifications for these go?

  • Avatar
    Danielle Morley

    Hello Carla,

    These emails are sent to the "problem Contact Email".  You can read more about how to change this here:  https://truecontext.zendesk.com/entries/21931407