Email authentication : SPF

In the previous post we described DNS lookup and the IPREV mechanisms. They were great SPAM killers but the rise of the Internet in the 2000s saw the emergence of increasingly complex email architectures. A new protocol became popular through an experimental RFC : the Send Policy Framework (SPF)

In 2014 the adoption of the RFC7208 definitely anchors SPF as a standard.

This is the second stage of your journey in the email authentication landscape.

What is SPF ?

Basically, the SPF framework specifies which servers are allowed to send mail from a specific domain name. If a spoofed server attempts to send an email, the receiving MTA checks the spoofer’s IP. As it will not be declared in the SPF framework of the original domain, the connection will be rejected.

That means that the SPF framework allows the Administrative Management Domains (ADMDs) to explicitly authorize hosts to use their domain in the “MAIL FROM” or “HELO” identities. The authorization list is published in the DNS records of the sender’s domain.

DNS Records

The type of a SPF record is TXT. There should be only one SPF record per domain.

Here is a basic SPF record example: “only servers in the range 123.123.123.0/24 and MTA (MX) are authorized to send emails from my domain mydomain.com. All other senders are considered unauthorized.”

mydomain.com.          TXT "v=spf1 +mx ip4:123.123.123.0/24 -all"Code language: JavaScript (javascript)

There is also a tilde version: ~all. It warns that other senders are not allowed, but must still be accepted. This “Soft Fail” statement was first introduced for testing purposes, but is now used by various hosting providers.

As a domain must not have multiple SPF records, multiple include must be declared into a unique DNS records.

mydomain.com	      TXT "v=spf1 include:domain.com include:otherdomain.com -all"Code language: JavaScript (javascript)

Please also notice that the total number of DNS lookups can’t exceed 10.

Great tools and articles about SPF compliancy can be found on various websites such as PowerDMARC or EasyDMARC.

SPF evaluation

HELO/EHLO versus MAIL FROM identity

The RFC7208 does not enforce a HELO/EHLO verification.

“It is RECOMMENDED that SPF verifiers not only check the “MAIL FROM” identity but also separately check the “HELO” identity […] Additionally, since SPF records published for “HELO” identities refer to a single host, when available, they are a very reliable source of host authorization status. Checking “HELO” before “MAIL FROM” is the RECOMMENDED sequence if both are checked.“

The RFC 5321 tends to normalize the HELO/EHLO arguments to represent the fully qualified domain name of the SMTP client. However the vSMTP SPF verifier is prepared for the identity to be an IP address literal or simply be malformed. In this case the “MAIL FROM” check occurs.

“SPF check can only be performed when the “HELO” string is a valid, multi-label domain name […] SPF verifiers must check the “MAIL FROM” identity if a “HELO” check either has not been performed or has not reached a definitive policy result.“

Note that RFC5321 allows the reverse-path to be null. In this case, the RFC7208 defines the “MAIL FROM” identity to be the mailbox composed of the local-part “postmaster” and the “HELO” identity.

Results

The vSMTP SPF verifier implements results semantically equivalent to the RFC.

Result	     Description
--------------------------------------------------------------------------------------------------------------------
none	     (a) no syntactically valid DNS domain name was extracted from the SMTP session that could be used as
             the one to be or (b) no SPF records were retrieved from the DNS.
neutral	     The ADMD has explicitly stated that it is not asserting whether the IP address is authorized.
pass	     The client is authorized to inject mail with the given identity.
fail	     The client is not authorized to use the domain in the given identity.
softfail     The host is probably not authorized but the ADMD has not published a stronger policy.
temperror    A transient (generally DNS) error while performing the check.
permerror    The domain’s published records (DNS) could not be correctly interpreted.
policy	     (NOT IMPLEMENTED) Code language: JavaScript (javascript)

None vs PermError

When a receiving MTA begins to perform SPF authentication on an email, it fetches all the DNS TXT records that begin with “v=spf1”. In case SPF is not configured for the sending domain, and no SPF record is found in the DNS, a None result is returned.

On the contrary, if multiple SPF records beginning with “v=spf1” are found to exist for the same domain, an SPF PermError result is returned.

Result headers

The RFC7208 recommends to store the SPF evaluation in a message header. Two options are available.

The “Received-SPF” header

This is the legacy header. It includes enough information to enable reconstruction of the SPF evaluation of the message.

Received-SPF: pass (mybox.example.org: domain of myname@example.com
  designates 192.0.2.1 as permitted sender)
  receiver=mybox.example.org; client-ip=192.0.2.1;
  envelope-from="myname@example.com"; helo=foo.example.com;Code language: JavaScript (javascript)

Please notice that the Received-SPF header field should be considered as trace field and therefore :

• It must appear above all other Received-SPF fields in the message.
• It should be prepended to the existing header, above the Received: field that is generated by the SMTP receiver.

The “Authentication-Results” header

This header is described in RFC8601 : Message Header Field for Indicating Message Authentication Status. It is designed to relay the result itself and related output details of likely use to end users.

Authentication-Results: example.com; spf=pass smtp.mailfrom=example.netCode language: HTTP (http)

Both headers are in common use. However, it is important to note that they were designed to serve slightly different purposes and therefore they should both be added.

vSMTP implementation

The vSMTP spf module describe all the required functions to handle SPF queries. To query a DNS server we have to build a DNS resolver. Let’s use the previous DNS resolvers we used in the lastest post.

const google_dns = dns::resolver(#{
    config: "google_tls",
});Code language: PHP (php)

A basic configuration.

fn on_helo(ctx) {

  // EHLO/HELO SPF check
  let helo_identity = spf::check_host(#{
    ip: ctx.client_ip,
    helo: ctx.helo,
    dns_resolver: global::dns_resolver
  });
  
  // Deny and close the connection if SPF result is not "pass"    
  if helo_identity != "pass" {
    return status::deny(`550 5.7.23 ${ctx.sender} is not allowed to send mail from ${ctx.client_ip}`)
  }

  ctx.store("helo", helo_identity);
  status::next()
}


fn on_mail_from(ctx) {

  // MAIL FROM SPF check
  let mail_from_identity = spf::check_host(#{
    ip: ctx.client_ip,
    helo: ctx.helo,
    mail_from: ctx.sender,
    dns_resolver: global::dns_resolver
  });

  if mail_from_identity != "pass" {
    status::deny(`550 5.7.23 ${ctx.sender} is not allowed to send mail from ${ctx.client_ip}`)
  } else {
    ctx.store("mail_from", mail_from_identity);
    status::next()
  }
}


fn on_pre_queue(ctx) {

  // Add authentication header
  auth::add_header(ctx, #{
    auth_serv_id: "mydomain.tld" // The domain name of the authentication server
  });
  status::next()
}Code language: JavaScript (javascript)

What’s next ?

With IPREV and SPF, we focused on sender verification. Now it’s time to ensure and verify email integrity. This is done via the DKIM protocol…

Stay tuned.

Email authentication #1 : DNS and IPREV

Email authentication protocols play crucial roles in securing email communication by verifying the authenticity of the sender and preventing email spoofing and phishing attacks. During this journey through the landscape of email authentication protocols, we will overfly protocols like IPREV or SPF but also DKIM, ARC and DMARC and the way to operate them using vSMTP.

Let’s take off.

Domain Name System : the common base

The first stage brings us in the 80s when the IETF (Internet Engineering Task Force) released the RFC 1034 (Request For Comment) : “DOMAIN NAMES – CONCEPTS AND FACILITIES”. This RFC aims to:

“[…]standardize the domain style names, their use for Internet mail and host address support, and the protocols and servers used to implement domain name facilities[…]“

Whatever the email authentication mechanism, it relies on the DNS (Domain Name System). If you have any doubts about how DNS works, there is an excellent article on the CloudFlare website.

IPREV : back to the 90s

In the 1990s, hackers quickly noticed the lack of SMTP sending server authentication, allowing them to send emails using a spoofed Internet domain name.

The first countermeasure deployed against Domain Name Spoofing was the Reverse DNS Lookup or PTR (Pointer) Record method. It was a simple but effective email authentication method.

By associating an IP address with a domain name by creating a reverse DNS record, when an email server receives an email, the server can perform a reverse DNS lookup to check if the sending IP address matches the domain’s DNS records.

Mail servers use reverse DNS to perform simple anti-spoofing checks but this method can be refined. The method is called “Forward-confirmed reverse DNS” or IPREV. This forward DNS lookup check that the reverse lookup matches the FQDN of the email header (aka seeing if it resolved back to the original IP number).

This mechanism has some limits. In some scenario i.e. when many MTAs are masked behind a unique IP address, reverse DNS query may return in a round-robin manner one of the server hostname.

vSMTP implementation

As always, we tried to make the work of administrators easier by adding functions that can be used directly in the configuration scripts of vSMTP.

The vSMTP dns and the iprev classes describe all the required functions to handle IPREV and DNS queries. To query a DNS server we have to build a DNS resolver. Let’s create a resolver that connects to Google DNS servers.

const google_dns = dns::resolver(#{
    config: "google_tls",
});Code language: PHP (php)

And the reverse lookup :

fn on_mail_from(ctx) {
  let result = dns::rlookup(#{
     dns_resolver: global::google_dns,
     ip: ctx.client_ip
  }); 

  // rlookup function returns an array of FQDN
  if result.is_empty() {
    return status::deny("550 5.7.25 Reverse DNS validation failed.");
  }

  // The first record must match the MAIL FROM domain
  if result[0] != ctx.sender.domain {
    return status::deny("550 5.7.25 Reverse DNS validation failed."); 
  }
 
  return status::next();
}
Code language: JavaScript (javascript)

This simple implementation should block most basic SPAM.

You should also use the iprev::check fonction to perform a forward-confirmed reverse DNS and, in case of failure, log the IP and close the connection.

fn on_connect(ctx) {
  let iprev_res = iprev::check(#{
     ip: ctx.client_ip,
     dns_resolver: global::google_dns
  });

  if iprev_res.value != "pass" {
    // Send a custom message to the log dispatcher using RFC 5424 "SD" style.
    log("info", `[netsec@vsmtp mechanism="iprev" ip="${ctx.client_ip}" domain="${ctx.sender.domain}"]`); 

    // Send an error code and close the connection
    return status::deny("550 5.7.25 Reverse DNS validation failed.");
  }

  return status::next();
}Code language: PHP (php)

Perhaps you may want to refine this implementation by fetching the log and updating a firewall rule to block this IP address or you may wish to add this pair (IP, domain) to an ephemeral cache to avoid saturating your DNS servers with requests… we will explain how to use the log dispatcher service and the native vSMTP plugins in upcoming posts.

Otherwise you can only store the iprev result in the context, so it can be used later to generate the “Authentication-Results” header field and continue processing.

fn on_connect(ctx) {
  let iprev_res = iprev::check(#{
     ip: ctx.client_ip,
     dns_resolver: global::google_dns
  });

  ctx.store(iprev_res);
  return status::next();
}Code language: PHP (php)

Imagination is the only limit to the power of vSMTP scripting.

What’s next ?

IPREV and reverse DNS lookup were great SPAM killers but the rise of the Internet in the 2000s saw the emergence of increasingly complex email architectures. A new protocol became popular through an experimental RFC : the Send Policy Framework (SPF)

In 2014 the adoption of the RFC 7208 definitely anchors SPF as a standard.

This will be the second stage of your journey.

Stay tuned.

Designed for cloud computing

There was not much movement happening in vSMTP for a few months, as we were working behind the scenes on the third version of vSMTP with a completely new architecture.

We rebuilt the server from the ground up, moving from a monolithic architecture to a micro-service architecture, using AMQP as the main way of communication between services.

A new architecture

Since then we did great optimizations, the transition to a micro-service architecture was the major shift in how vSMTP version 3 is structured. This change involved breaking down the server into smaller, more manageable services, each responsible for specific functions.

The micro-service architecture offers benefits such as scalability, flexibility, and easier maintenance.

RabbitMQ and AMQP

Using AMQP (Advanced Message Queuing Protocol) and RabbitMQ as vSMTP messaging infrastructure was an obvious choice.

RabbitMQ acts as a message broker, facilitating the exchange of messages between different vSMTP services. It provides features such as message queuing, routing, and load balancing, which improve the reliability and scalability of vSMTP.

Microservices, AMQP and cloud computing

Microservices architecture and the use of Advanced Message Queuing Protocol are well suited to cloud computing for several reasons, as they align with the key principles and benefits of cloud-based applications:

• Scalability and elasticity
• Resource allocation and efficiency
• High availability and fault tolerance
• Global reach

In summary, microservices architecture and cloud computing naturally complement each other due to the alignment of their fundamental principles and capabilities.

The combination of vSMTP’s new internal architecture with the scalability, flexibility, and managed services of cloud platforms enables messaging service providers to build and operate modern, resilient, and efficient infrastructures that can meet the demands today’s dynamic and evolving business environments.

You can right now download the release candidate on GitHub. Our teams will be happy to guide you during your installation.

Stay tuned. Join us on Discord, X and LinkedIn.

A rocket is launched

350,000,000,000

An astronomical number – 350 billion emails per day routed and analyzed by software built in the 90s.

The numerous energy and environmental crises have made us aware that any new IT product must be designed with the major objective of reducing the environmental footprint. It is on this premise that the vSMTP project was born in 2020.

Since its first releases in 2021, vSMTP is certainly one of the fastest MTAs in the world.

Since then we did great optimizations, but, when considering an MTA in 2023, its intrinsic performance is not the only quality to take into account. It must be easily usable and configurable. The vSMTP version 2 specifications met these criteria by providing an advanced scripting language as well as a very easy to use rules system.

Version 3

We wanted to go further, and design a new version capable of integrating into a world where cloud computing reigns supreme and where companies’ commitment to CSR has become mainstream.

The viridIT teams have worked hard to answer them and I am proud to announce the release of version 3 of vSMTP at the beginning of October. This major release brings a completely new architecture based on microservices.

You can right now download the release candidate on GitHub. Our teams will be happy to guide you during your installation.

Stay tuned. Join us on Discord, X and LinkedIn.