Reply categories and the classification process
Siftrock's classifies replies using machine learning techniques and a proprietary natural language engine that continually gets better over time as it sees more emails.
This type indicates what's commonly referred to as a soft-bounce. The recipients email server has responded with an automated message indicating the email couldn't be delivered. Sometimes a reason is given - such as a full inbox.
Occasionally people change email addresses. For example, a company might re-brand and take a different domain name. Another example is someone taking a married name. These replies indicate this type of change and will include the original recipient as a new contact with the new information.
Out of Office
The response indicated this person is out of the office. There was no detail provided so it's unclear whether this is vacation, general leave, maternity leave, or other type of leave.
The system identifies non-automated responses as human. These are particularly important because it's likely that a person responding directly to your email is expecting a reply back in a timely manner.
The system can distinguish between a reply and a direct inquiry. If a person emails your marketing alias directly, we'll classify that as human direct.
An auto-response indicated this person has left the company. New contacts discovered in these emails often name a successor or another person in the company as a new point of contact.
Over time we've seen an increasing number of auto responses from spam shield services such as Spam Arrest. Individuals using these services will not receive your emails without a manual validation step.
Identifying these is important and allows you to validate yourself as a sender. This only needs to be done once per reply-to address and ensures your emails won't get blocked going forward.
Some email addresses have a continual automated reply configured. Companies do this for a variety of reasons. One example is the main support email address will often respond with an immediate "thank you for your inquiry" email.
These are classified as system responses since you are not likely to ever find a human reading these emails or responding directly.
Despite our best efforts sometimes emails can't be correctly classified into the supported categories. The most common cause of this is emails in a language other than English. Another common case is emails sent with no content in the email body.
Human response that indicate a request to be unsubscribed from future emails. Many users will ignore your unsubscribe links and instead respond to your emails asking to be removed from your list. We find all kind of unsubscribe language so you can automatically update these contacts to exclude from future marketing.
These are out of office emails that indicate an individual is out specifically on vacation or holiday.
Accuracy of Classification
We strive to continually improve accuracy but it's important to understand that natural language processing is an imperfect science. Here is a focused example to help illustrate:
The word "retire" appears in various forms of automated reply. Examples we've seen include:
- "I have retired from..."
- "Bob Smith retired from our company last month..."
- "I'm happy to announce my retirement..."
- "I will retire from my position next month..."
Having seen so many of these types of emails the natural language engine recognizes forms of the word "retire" only appear in emails where someone has left (or is leaving) the company. Thus emails that look like this are classified as "Left Company".
This is a real email that came through for one of our customers:
- "I have retired this email address and will be using this new one going forward..."
The human brain is capable of easily recognizing that this is actually a "Changed my email address" reply. However, the natural language engine doesn't have enough context or examples to understand this. Thus it gets incorrectly classified as "Left Company".
In processing more than 2 million replies we've only seen this language used once. Statistically we are unlikely to ever have enough examples to appropriately train the system to understand the difference.