Asynchronous Jobs

This page describes the asynchronous background job process used to handle long running tasks in the PayRun.io API.

What are jobs?

Jobs are tasks executed asynchronously. The driver behind the use of jobs is scalability.

PayRun.io is designed as an enterprise level payroll processing solution, capable of handling payolls of many thousand employees.

In order to handle large volumes of processor intensive payroll calculations and other long running tasks, without impacting overall system performance, payroll calculations are performed by distributed worker services.

Job instructions are enqueued by the end user and then de-queued by the worker processes (AKA Queue Workers). The worker updates the status of the job as it processes the instruction and the end user monitors the status.

Job Queue

Items added to the queue are processed in the order they have been submitted. Jobs for the same underlying entity are only released when the proceeding job has completed. For example, if you submit several Pay Run Job Instructions for the same pay schedule, the jobs will only be processed one at a time in the sequence of insertion.

Inserting a Job

When inserting a new job instruction, you receive a link to the jobs information page. Using this link, it is possible to monitor the job progress.

curl -X POST \
  'https://api.test.payrun.io/jobs/payruns' \
  -H 'Accept: application/xml' \
  -H 'Api-Version: default' \
  -H 'Authorization: {OAuthHeader}' \
  -H 'Cache-Control: no-cache' \
  -H 'Content-type: application/xml' \
  -d '<?xml version="1.0"?>
<PayRunJobInstruction xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <HoldingDate xsi:nil="true" />
  <PaymentDate>2017-04-30</PaymentDate>
  <StartDate>2017-04-01</StartDate>
  <EndDate>2017-04-30</EndDate>
  <PaySchedule href="/Employer/ER001/PaySchedule/SCH001" />
  <IsSupplementary>false</IsSupplementary>
</PayRunJobInstruction>'

curl -X POST \
  'https://api.test.payrun.io/jobs/payruns' \
  -H 'Accept: application/json' \
  -H 'Api-Version: default' \
  -H 'Authorization: {OAuthHeader}' \
  -H 'Cache-Control: no-cache' \
  -H 'Content-type: application/json' \
  -d '{
  "PayRunJobInstruction": {
    "@xmlns:xsi": "http://www.w3.org/2001/XMLSchema-instance",
    "HoldingDate": null,
    "PaymentDate": "2017-04-30",
    "StartDate": "2017-04-01",
    "EndDate": "2017-04-30",
    "PaySchedule": {
      "@href": "/Employer/ER001/PaySchedule/SCH001"
    },
    "IsSupplementary": "false",
    "Employees": null
  }
}'

Response Example

<?xml version="1.0"?>
<Link 
    xmlns:xsd="http://www.w3.org/2001/XMLSchema" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    title="Pay Run: [/2017-04-30]" 
    href="/jobs/payruns/26ece157-e199-4e22-8495-36dcd53388c2/info" 
    rel="JobInfo" 
    />

{
  "Link": {
    "@title": "Pay Run: [/2017-04-30]",
    "@href": "/jobs/payruns/26ece157-e199-4e22-8495-36dcd53388c2/info",
    "@rel": "JobInfo"
  }
}

Querying the Job Status

To query the current status of a job you perform a GET request to the /info endpoint returned from the a successful job enqueue (as above).

curl -X GET \
  'https://api.test.payrun.io/jobs/payruns/88738be7-e499-48ba-a4ba-c3467aaafd9b/info' \
  -H 'Accept: application/xml' \
  -H 'Api-Version: default' \
  -H 'Authorization: {OAuthHeader}' \
  -H 'Cache-Control: no-cache' \
  -H 'Content-type: application/xml'

curl -X GET \
  'https://api.test.payrun.io/jobs/payruns/88738be7-e499-48ba-a4ba-c3467aaafd9b/info' \
  -H 'Accept: application/json' \
  -H 'Api-Version: default' \
  -H 'Authorization: {OAuthHeader}' \
  -H 'Cache-Control: no-cache' \
  -H 'Content-type: application/json'

Response Example

<?xml version="1.0"?>
<JobInfo xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <JobId>26ece157-e199-4e22-8495-36dcd53388c2</JobId>
    <Created>2017-10-24T16:51:12.7166667</Created>
    <LastUpdated>2017-10-24T16:51:13.1933333</LastUpdated>
    <JobStatus>InProgress</JobStatus>
    <Progress>0.520</Progress>
    <HoldingDate xsi:nil="true" />
</JobInfo>

{
  "JobInfo": {
    "@xmlns:xsi": "http://www.w3.org/2001/XMLSchema-instance",
    "JobId": "26ece157-e199-4e22-8495-36dcd53388c2",
    "Created": "2017-10-24T16:51:12.7166667",
    "LastUpdated": "2017-10-24T16:51:13.1933333",
    "JobStatus": "InProgress",
    "Progress": "0.520",
    "HoldingDate": null
  }
}

The response indicates that job is in progress and 52% complete.

Deleting a Job

You are free to delete any job that is not In Progress.

curl -X DELETE \
  'https://api.test.payrun.io/Jobs/PayRuns/6ee0ef5f-f7be-44e7-85ef-4a35150490d1' \
  -H 'Accept: application/xml' \
  -H 'Api-Version: default' \
  -H 'Authorization: {OAuthHeader}' \
  -H 'Cache-Control: no-cache' \
  -H 'Content-type: application/xml'

curl -X DELETE \
  'https://api.test.payrun.io/Jobs/PayRuns/6ee0ef5f-f7be-44e7-85ef-4a35150490d1' \
  -H 'Accept: application/json' \
  -H 'Api-Version: default' \
  -H 'Authorization: {OAuthHeader}' \
  -H 'Cache-Control: no-cache' \
  -H 'Content-type: application/json'

Queue Workers

The queue worker processes are specialists. Each type of worker handles a specific type of job instruction. This allows us to scale the support for certain job types independently and makes the workers simplier to develop.

Job Instructions

Each type of job has a unique instruction. The instruction includes all the nescessary details needed by the worker to perform the required actions.

Holding Date

All job instructions include an optional property: Holding Date
By setting the holding date you can defer execution of the job until a future point in time.

Job Status

Jobs can be any of the below states.

Pending
The job is on the queue waiting to be picked up by a worker.
In Progress
The job has been selected by a worker and is actively being worked on.
Failed
The job completed with errors.
Success
The job completed without errors.

Job Errors

If any exceptions are encountered during the job processing, the error messages are written back to the job errors collection. You can view the errors collection by calling the job information page.

curl -X GET \
  'https://api.test.payrun.io/jobs/payruns/26ece157-e199-4e22-8495-36dcd53388c2/info' \
  -H 'Accept: application/xml' \
  -H 'Api-Version: default' \
  -H 'Authorization: {OAuthHeader}' \
  -H 'Cache-Control: no-cache' \
  -H 'Content-type: application/xml'

curl -X GET \
  'https://api.test.payrun.io/jobs/payruns/26ece157-e199-4e22-8495-36dcd53388c2/info' \
  -H 'Accept: application/json' \
  -H 'Api-Version: default' \
  -H 'Authorization: {OAuthHeader}' \
  -H 'Cache-Control: no-cache' \
  -H 'Content-type: application/json'

Response Example

<?xml version="1.0"?>
<JobInfo xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <JobId>26ece157-e199-4e22-8495-36dcd53388c2</JobId>
    <Created>2017-10-24T16:51:12.7166667</Created>
    <LastUpdated>2017-10-24T16:51:13.1933333</LastUpdated>
    <JobStatus>Failed</JobStatus>
    <Progress>1.000</Progress>
    <HoldingDate xsi:nil="true" />
    <Errors>
        <Error>Data Integrity Violation Detected. Unable to perform requested 'Insert' action on entity 'PayRun'.</Error>
        <Error>[DIR1205] - Cannot insert pay run. Incorrect period start date specified. Pay run start date must directly follow the proceeding pay run end date. Expected start date 2016-05-01 but found 2017-04-01</Error>
    </Errors>
</JobInfo>

{
  "JobInfo": {
    "@xmlns:xsi": "http://www.w3.org/2001/XMLSchema-instance",
    "JobId": "26ece157-e199-4e22-8495-36dcd53388c2",
    "Created": "2017-10-24T16:51:12.7166667",
    "LastUpdated": "2017-10-24T16:51:13.1933333",
    "JobStatus": "Failed",
    "Progress": "1.000",
    "HoldingDate": null,
    "Errors": {
      "Error": [
        "Data Integrity Violation Detected. Unable to perform requested 'Insert' action on entity 'PayRun'.",
        "[DIR1205] - Cannot insert pay run. Incorrect period start date specified. Pay run start date must directly follow the proceeding pay run end date. Expected start date 2016-05-01 but found 2017-04-01"
      ]
    }
  }
}