Handling and Recovering from failed Queueable executions

How to handle Queueable executions in order to track errors and launch again.

When using Queuable Interface to execute business logic or whatever you need, I found no way to know about a possible fail of the execution, except for the «Apex Exception Email» option, but it’s not very useful, yes… there may be some stacktrace, but I miss some execution log, related record ids and any other business data which helps you to retake the process on the same point it failed.

So, I think it could be a good point for Salesforce to add some recovering features to the Apex Jobs, meanwhile… I will try to do it by myself

In the same way aspects work in the Java world I will wrap the business Queuable execute method with some code making the magic.

Vegeta's_big_bang_attack

This code will catch any Exception thrown by the business Queueable execute and will log it to a Custom Object, I named Queue Monitor, serializing the Queuable Job using JSON, this serialization will allow us to launch again the job with the same parameters it failed, also it will be easy to copy the failure’s records to a Full Sandbox and reproduce it.

Hope you find this useful, enjoy with Force.com.

global class QueueableHandler implements Queueable, Database.AllowsCallouts {
    private Object job {get; set;}
    private String jobClassName {get; set;}
    private Queue_Monitor__c monitor {get; set;}
    public String jobId {get; set;}
    /**
     * Object names
     */
    public static final List<String> OBJECT_NAMES = new List<String> {'Case', 'Opportunity', 'Account', 'Contact'};

    /**
     * Use this constructor for first Queue
     */
    public QueueableHandler(Object qJob, System.Type qJobClass) {
        this(qJob, qJobClass, null);
    }

    /**
     * Use this constructor for retiries, this will update the Queue_Monitor__c record
     */
    public QueueableHandler(Object qJob, System.Type qJobClass, Queue_Monitor__c qm) {
        this.job = qJob;
        this.jobClassName = qJobClass.getName();
        this.monitor = qm;
        this.jobId = System.enqueueJob(this);
    }

    /**
     * Use this webservice method to retry Jobs from Queue_Monitor__c records
     * @param monitorId
     * @return new Queueable Job Id
     */
    webservice static String retryJob(String monitorId) {
        Queue_Monitor__c monitor = [SELECT JSON_Job__c, Job_Class__c, Last_try__c, Job_Id__c, Status__c FROM Queue_Monitor__c WHERE Id = :monitorId];
        if(monitor.Status__c == 'Queued') {
            return monitor.Job_Id__c;
        }
        System.Type clazz = Type.forName(monitor.Job_Class__c);
        monitor.Status__c = 'Queued';
        update monitor;
        // Apply regexp when deserialize to fix picklist fields
        QueueableHandler qh = new QueueableHandler((Queueable)JSON.deserialize(monitor.JSON_Job__c.replaceAll('\\{"value":"([\\w\\s0-9]+)"\\}', '"$1"'), clazz), clazz, monitor);
        return qh.jobId;
    }

    public void execute(QueueableContext context) {
        try {
            if(monitor != null) {
                monitor.Last_try__c = Datetime.now();
                monitor.Job_Id__c = context.getJobId();
                monitor.User__c = UserInfo.getUserId();
            }
            ((Queueable)job).execute(context);
            if(monitor != null) {
                monitor.Status__c = 'Completed';
                update monitor;
            }
        } catch (Exception ex) {
            Queue_Monitor__c qm  = monitor;
            if(qm == null) {
                String jobString = JSON.serialize(job);
                String recordId = '';
                String objectName = '';
                try {
                    for(String objName : OBJECT_NAMES) {
                        objectName = objName;
                        if(jobString.contains('/sobjects/' + objName + '/')) {
                            recordId = jobString.replaceAll('.*/sobjects/' + objName + '/(\\w{15,18})\\".*', '$1');
                            break;
                        }
                    }
                } catch(System.LimitException limitEx) {
                    //Catch System.LimitException: Regex too complicated
                    recordId = 'FailedToObtainId';
                }
                qm  = new Queue_Monitor__c(JSON_Job__c = jobString,
                                           Job_Class__c = jobClassName,
                                           User__c = UserInfo.getUserId(),
                                           Job_Id__c = context.getJobId(),
                                           Record_Id__c = recordId,
                                           Object__c = objectName,
                                           Status__c = 'Failed',
                                           Exception_Text__c = String.format('Exception: {0}\nMessage: {1}\nLine number:{2}\nStacktrace: {3}',
                                                   new List<String> { ex.getTypeName(),
                                                           ex.getMessage(),
                                                           '' + ex.getLineNumber(),
                                                           ex.getStackTraceString()
                                                                    } ));
                insert qm;
            } else {
                qm.Status__c = 'Failed';
                update qm;
            }
        }
    }
}

Code Highlights

  • webservice static retryJob(String monitorId). Exposing retries via webservice allow us to launch from a Custom Button using some javascript.
  • replaceAll(‘\\{«value»:»(.+)»\\}’, ‘»$1″‘) I found some extrange behaviour on JSON Serialization, making deserialization to fail, this replacement fix it, see JSON Serialization & Deserialization
  • System.Type clazz = Type.forName(monitor.Job_Class__c) Dynamic class loading, thanks to Alex Tennant (Salesforce MVP) his mention to Dependency Injection unleashed me.
  • public static final List<String> OBJECT_NAMES Customize the Objects or Custom Objects to lookup for Record Id.