Exercise 21: Identifying Open Relays in SMTP Servers

Objective

Test an SMTP server for open relay vulnerabilities and secure it by disabling unauthorized email relaying.

Scenario

As a security analyst, you are tasked with verifying that the organization’s SMTP server is not vulnerable to open relay attacks. An open relay allows unauthorized users to send emails through your mail server, potentially leading to spam distribution and blacklisting. In this exercise, you’ll configure an SMTP server, test for open relay vulnerabilities using telnet, and implement security measures to prevent unauthorized relaying.

⚠️ Important: This exercise should be performed in a legal and controlled lab environment. Unauthorized use of SMTP servers is illegal and unethical.

Lab Instructions

Step 1: Set Up a Local SMTP Server

Install Postfix as the SMTP server on a Linux machine:

sudo apt update 
sudo apt install postfix -y

Select Internet Site when prompted and configure the mail server domain (e.g., example.com).

Step 2: Verify SMTP Server is Running

Confirm that Postfix is active:

sudo systemctl status postfix

Ensure port 25 is open and listening:

sudo netstat -tuln | grep :25

Step 3: Attempt to Send an Unauthorized Email Using Telnet

Connect to the SMTP server using telnet:

telnet <smtp-server-ip> 25

Enter the following SMTP commands to simulate an email relay:

HELO attacker.com
MAIL FROM:<attacker@malicious.com>
RCPT TO:<victim@target.com>
DATA
Subject: Test Open Relay

This is a test email.
.
QUIT

Expected Result:

If the server accepts the email for delivery, it is an open relay.

If the server rejects the email, it is properly secured.

Step 4: Secure the SMTP Server (Disable Open Relay)

Edit Postfix’s configuration file:

sudo nano /etc/postfix/main.cf

Ensure the following lines are set to restrict relaying:

smtpd_relay_restrictions = permit_mynetworks, reject_unauth_destination 
mynetworks = 127.0.0.0/8 [::1]/128

Apply the configuration changes:

sudo systemctl restart postfix

Step 5: Re-Test for Open Relay

Repeat the telnet test:

telnet <smtp-server-ip> 25

Expected Result: The server should now reject the email with an error like:

554 Relay access denied

Solution & Explanation

What is an Open Relay?

• An open relay allows any external user to send emails through the SMTP server.
• Attackers exploit open relays to send spam or phishing emails, risking domain blacklisting.

Why It Happens

• Misconfigured Postfix or other SMTP servers can allow relaying from unauthorized sources.
• Incorrect mynetworks or smtpd_relay_restrictions settings.

Securing the SMTP Server

• permit_mynetworks: Allows relaying only from trusted IP addresses.
• reject_unauth_destination: Blocks unauthorized delivery attempts.
• Firewall Rules: Block SMTP access (port 25) from external networks if not needed.

Testing & Verification

Confirm that the server rejects unauthorized emails after securing it.

Verify legitimate users on the internal network can still send emails.

Monitor mail logs:

sudo tail -f /var/log/mail.log

Security Best Practices

1. Restrict Relaying: Allow only trusted networks to relay emails.
2. Enable Authentication: Require SMTP authentication for email submissions.
3. Implement SPF, DKIM, and DMARC: Strengthen email security policies.
4. Monitor Logs: Regularly review mail logs for suspicious activity.
5. Use Rate Limiting: Throttle email sending to prevent abuse.

Additional Script (Optional)

Automate SMTP security configuration:

#!/bin/bash
# Secure Postfix to prevent open relay
sudo postconf -e "smtpd_relay_restrictions=permit_mynetworks,reject_unauth_destination"
sudo postconf -e "mynetworks=127.0.0.0/8 [::1]/128"
sudo systemctl restart postfix

echo "Postfix has been secured against open relay."

Run the script:

chmod +x secure_postfix.sh
sudo ./secure_postfix.sh

Conclusion

In this exercise, you tested an SMTP server for open relay vulnerabilities using telnet. You secured the server by configuring relay restrictions and verified the fix. Properly configuring SMTP servers is critical to preventing unauthorized email relaying and protecting against spam and domain blacklisting.