A migration challenge

Scaling business to a best-in-class digital position often involves a complete IT infrastructure transformation.
These transformation projects are therefore often seen as highly disruptive, costly and presenting challenges that take time to resolve or are difficult to overcome. According to Forbes, 84% of digital transformation efforts fail due to the magnitude of the challenges posed.

As a result, decision-makers and users are often hesitant to engage in such a transformation. As such, they often find themselves trying to overthink the problem, following change processes that require everything to stop and start from scratch.

If your instinct makes you anxious about new technologies and strategies, then you are not alone.

To address this issue, at viridIT we offer a non-disruptive approach, based on outsourcing standards: the T&T (Transition and Transformation) method.

The transition step

The transition phase is a smooth migration, consisting of replacing certain components with a constant functional scope. The customer’s infrastructure is not impacted.

The strength of vSMTP is that it can easily adapt and mimic the behavior of its competitors.

In collaboration with the client teams, we analyze the architecture of the current system as well as the incoming and outgoing flows from the client’s backend (logs, SMTP, HTTP, etc.)

Next comes the vSMTP configuration phase.
In order to minimize the impact on the client’s IT backend, our teams remotely build a tailor-made vSMTP on viridIT infrastructures.

This period lasts on average 3 to 4 weeks and only requires a few man days for the client.

Getting started

As with any new tool, customer teams must understand and master vSMTP. This way we distribute the necessary support and training during this period.

And it is by acquiring mastery of vSMTP that customer teams will begin to design and imagine another way of operating, using the power of vSMTP.

The transformation step

A common mistake is to view transformation as a process that must begin and end at some point.
It is a process that occurs in manageable stages rather than a significant and immediate need to move to its final transformed state.
Transformation should not be seen as a final step but rather as a process of continuous improvement.

The better the tools, the better the business

Although they involve different mechanisms of change, digital transformation and transition are essential to the future success of your business.

Technology is constantly evolving, becoming more functional and encompassing new business needs.
This highlights the shortcomings of older, outdated technology that makes business processes slower and more expensive than competitors.

Upgrading outdated software may seem like a basic plan, but if not considered, it can significantly hamper your business processes. vSMTP offers new ways for your business to operate and ensure a successful future.
For more information on how to improve and transform your processes, contact viridIT today.

Email authentication : SPF

In the previous post we described DNS lookup and the IPREV mechanisms. They were great SPAM killers but the rise of the Internet in the 2000s saw the emergence of increasingly complex email architectures. A new protocol became popular through an experimental RFC : the Send Policy Framework (SPF)

In 2014 the adoption of the RFC7208 definitely anchors SPF as a standard.

This is the second stage of your journey in the email authentication landscape.

What is SPF ?

Basically, the SPF framework specifies which servers are allowed to send mail from a specific domain name. If a spoofed server attempts to send an email, the receiving MTA checks the spoofer’s IP. As it will not be declared in the SPF framework of the original domain, the connection will be rejected.

That means that the SPF framework allows the Administrative Management Domains (ADMDs) to explicitly authorize hosts to use their domain in the “MAIL FROM” or “HELO” identities. The authorization list is published in the DNS records of the sender’s domain.

DNS Records

The type of a SPF record is TXT. There should be only one SPF record per domain.

Here is a basic SPF record example: “only servers in the range 123.123.123.0/24 and MTA (MX) are authorized to send emails from my domain mydomain.com. All other senders are considered unauthorized.”

mydomain.com.          TXT "v=spf1 +mx ip4:123.123.123.0/24 -all"Code language: JavaScript (javascript)

There is also a tilde version: ~all. It warns that other senders are not allowed, but must still be accepted. This “Soft Fail” statement was first introduced for testing purposes, but is now used by various hosting providers.

As a domain must not have multiple SPF records, multiple include must be declared into a unique DNS records.

mydomain.com	      TXT "v=spf1 include:domain.com include:otherdomain.com -all"Code language: JavaScript (javascript)

Please also notice that the total number of DNS lookups can’t exceed 10.

Great tools and articles about SPF compliancy can be found on various websites such as PowerDMARC or EasyDMARC.

SPF evaluation

HELO/EHLO versus MAIL FROM identity

The RFC7208 does not enforce a HELO/EHLO verification.

“It is RECOMMENDED that SPF verifiers not only check the “MAIL FROM” identity but also separately check the “HELO” identity […] Additionally, since SPF records published for “HELO” identities refer to a single host, when available, they are a very reliable source of host authorization status. Checking “HELO” before “MAIL FROM” is the RECOMMENDED sequence if both are checked.“

The RFC 5321 tends to normalize the HELO/EHLO arguments to represent the fully qualified domain name of the SMTP client. However the vSMTP SPF verifier is prepared for the identity to be an IP address literal or simply be malformed. In this case the “MAIL FROM” check occurs.

“SPF check can only be performed when the “HELO” string is a valid, multi-label domain name […] SPF verifiers must check the “MAIL FROM” identity if a “HELO” check either has not been performed or has not reached a definitive policy result.“

Note that RFC5321 allows the reverse-path to be null. In this case, the RFC7208 defines the “MAIL FROM” identity to be the mailbox composed of the local-part “postmaster” and the “HELO” identity.

Results

The vSMTP SPF verifier implements results semantically equivalent to the RFC.

Result	     Description
--------------------------------------------------------------------------------------------------------------------
none	     (a) no syntactically valid DNS domain name was extracted from the SMTP session that could be used as
             the one to be or (b) no SPF records were retrieved from the DNS.
neutral	     The ADMD has explicitly stated that it is not asserting whether the IP address is authorized.
pass	     The client is authorized to inject mail with the given identity.
fail	     The client is not authorized to use the domain in the given identity.
softfail     The host is probably not authorized but the ADMD has not published a stronger policy.
temperror    A transient (generally DNS) error while performing the check.
permerror    The domain’s published records (DNS) could not be correctly interpreted.
policy	     (NOT IMPLEMENTED) Code language: JavaScript (javascript)

None vs PermError

When a receiving MTA begins to perform SPF authentication on an email, it fetches all the DNS TXT records that begin with “v=spf1”. In case SPF is not configured for the sending domain, and no SPF record is found in the DNS, a None result is returned.

On the contrary, if multiple SPF records beginning with “v=spf1” are found to exist for the same domain, an SPF PermError result is returned.

Result headers

The RFC7208 recommends to store the SPF evaluation in a message header. Two options are available.

The “Received-SPF” header

This is the legacy header. It includes enough information to enable reconstruction of the SPF evaluation of the message.

Received-SPF: pass (mybox.example.org: domain of myname@example.com
  designates 192.0.2.1 as permitted sender)
  receiver=mybox.example.org; client-ip=192.0.2.1;
  envelope-from="myname@example.com"; helo=foo.example.com;Code language: JavaScript (javascript)

Please notice that the Received-SPF header field should be considered as trace field and therefore :

• It must appear above all other Received-SPF fields in the message.
• It should be prepended to the existing header, above the Received: field that is generated by the SMTP receiver.

The “Authentication-Results” header

This header is described in RFC8601 : Message Header Field for Indicating Message Authentication Status. It is designed to relay the result itself and related output details of likely use to end users.

Authentication-Results: example.com; spf=pass smtp.mailfrom=example.netCode language: HTTP (http)

Both headers are in common use. However, it is important to note that they were designed to serve slightly different purposes and therefore they should both be added.

vSMTP implementation

The vSMTP spf module describe all the required functions to handle SPF queries. To query a DNS server we have to build a DNS resolver. Let’s use the previous DNS resolvers we used in the lastest post.

const google_dns = dns::resolver(#{
    config: "google_tls",
});Code language: PHP (php)

A basic configuration.

fn on_helo(ctx) {

  // EHLO/HELO SPF check
  let helo_identity = spf::check_host(#{
    ip: ctx.client_ip,
    helo: ctx.helo,
    dns_resolver: global::dns_resolver
  });
  
  // Deny and close the connection if SPF result is not "pass"    
  if helo_identity != "pass" {
    return status::deny(`550 5.7.23 ${ctx.sender} is not allowed to send mail from ${ctx.client_ip}`)
  }

  ctx.store("helo", helo_identity);
  status::next()
}


fn on_mail_from(ctx) {

  // MAIL FROM SPF check
  let mail_from_identity = spf::check_host(#{
    ip: ctx.client_ip,
    helo: ctx.helo,
    mail_from: ctx.sender,
    dns_resolver: global::dns_resolver
  });

  if mail_from_identity != "pass" {
    status::deny(`550 5.7.23 ${ctx.sender} is not allowed to send mail from ${ctx.client_ip}`)
  } else {
    ctx.store("mail_from", mail_from_identity);
    status::next()
  }
}


fn on_pre_queue(ctx) {

  // Add authentication header
  auth::add_header(ctx, #{
    auth_serv_id: "mydomain.tld" // The domain name of the authentication server
  });
  status::next()
}Code language: JavaScript (javascript)

What’s next ?

With IPREV and SPF, we focused on sender verification. Now it’s time to ensure and verify email integrity. This is done via the DKIM protocol…

Stay tuned.

Email authentication #1 : DNS and IPREV

Email authentication protocols play crucial roles in securing email communication by verifying the authenticity of the sender and preventing email spoofing and phishing attacks. During this journey through the landscape of email authentication protocols, we will overfly protocols like IPREV or SPF but also DKIM, ARC and DMARC and the way to operate them using vSMTP.

Let’s take off.

Domain Name System : the common base

The first stage brings us in the 80s when the IETF (Internet Engineering Task Force) released the RFC 1034 (Request For Comment) : “DOMAIN NAMES – CONCEPTS AND FACILITIES”. This RFC aims to:

“[…]standardize the domain style names, their use for Internet mail and host address support, and the protocols and servers used to implement domain name facilities[…]“

Whatever the email authentication mechanism, it relies on the DNS (Domain Name System). If you have any doubts about how DNS works, there is an excellent article on the CloudFlare website.

IPREV : back to the 90s

In the 1990s, hackers quickly noticed the lack of SMTP sending server authentication, allowing them to send emails using a spoofed Internet domain name.

The first countermeasure deployed against Domain Name Spoofing was the Reverse DNS Lookup or PTR (Pointer) Record method. It was a simple but effective email authentication method.

By associating an IP address with a domain name by creating a reverse DNS record, when an email server receives an email, the server can perform a reverse DNS lookup to check if the sending IP address matches the domain’s DNS records.

Mail servers use reverse DNS to perform simple anti-spoofing checks but this method can be refined. The method is called “Forward-confirmed reverse DNS” or IPREV. This forward DNS lookup check that the reverse lookup matches the FQDN of the email header (aka seeing if it resolved back to the original IP number).

This mechanism has some limits. In some scenario i.e. when many MTAs are masked behind a unique IP address, reverse DNS query may return in a round-robin manner one of the server hostname.

vSMTP implementation

As always, we tried to make the work of administrators easier by adding functions that can be used directly in the configuration scripts of vSMTP.

The vSMTP dns and the iprev classes describe all the required functions to handle IPREV and DNS queries. To query a DNS server we have to build a DNS resolver. Let’s create a resolver that connects to Google DNS servers.

const google_dns = dns::resolver(#{
    config: "google_tls",
});Code language: PHP (php)

And the reverse lookup :

fn on_mail_from(ctx) {
  let result = dns::rlookup(#{
     dns_resolver: global::google_dns,
     ip: ctx.client_ip
  }); 

  // rlookup function returns an array of FQDN
  if result.is_empty() {
    return status::deny("550 5.7.25 Reverse DNS validation failed.");
  }

  // The first record must match the MAIL FROM domain
  if result[0] != ctx.sender.domain {
    return status::deny("550 5.7.25 Reverse DNS validation failed."); 
  }
 
  return status::next();
}
Code language: JavaScript (javascript)

This simple implementation should block most basic SPAM.

You should also use the iprev::check fonction to perform a forward-confirmed reverse DNS and, in case of failure, log the IP and close the connection.

fn on_connect(ctx) {
  let iprev_res = iprev::check(#{
     ip: ctx.client_ip,
     dns_resolver: global::google_dns
  });

  if iprev_res.value != "pass" {
    // Send a custom message to the log dispatcher using RFC 5424 "SD" style.
    log("info", `[netsec@vsmtp mechanism="iprev" ip="${ctx.client_ip}" domain="${ctx.sender.domain}"]`); 

    // Send an error code and close the connection
    return status::deny("550 5.7.25 Reverse DNS validation failed.");
  }

  return status::next();
}Code language: PHP (php)

Perhaps you may want to refine this implementation by fetching the log and updating a firewall rule to block this IP address or you may wish to add this pair (IP, domain) to an ephemeral cache to avoid saturating your DNS servers with requests… we will explain how to use the log dispatcher service and the native vSMTP plugins in upcoming posts.

Otherwise you can only store the iprev result in the context, so it can be used later to generate the “Authentication-Results” header field and continue processing.

fn on_connect(ctx) {
  let iprev_res = iprev::check(#{
     ip: ctx.client_ip,
     dns_resolver: global::google_dns
  });

  ctx.store(iprev_res);
  return status::next();
}Code language: PHP (php)

Imagination is the only limit to the power of vSMTP scripting.

What’s next ?

IPREV and reverse DNS lookup were great SPAM killers but the rise of the Internet in the 2000s saw the emergence of increasingly complex email architectures. A new protocol became popular through an experimental RFC : the Send Policy Framework (SPF)

In 2014 the adoption of the RFC 7208 definitely anchors SPF as a standard.

This will be the second stage of your journey.

Stay tuned.