I’m getting hundreds or thousands of these lines in my logs each day (with different IP addresses, of course)
DNS list lookup defer (probably timeout) for 123.123.123.123.zen.spamhaus.org: assumed not in list: 1 Time(s)
What’s going wrong?
It looks like the server can’t do lookups against the Zen Spamhaus list.
Have a look at The Spamhaus Project - Frequently Asked Questions (FAQ) for info troubleshooting.
As far as I can make out, that is telling me to use
dig +short 80.68.80.24 1.0.0.127.zen.spamhaus.org
and
dig +short 80.68.80.24 2.0.0.127.zen.spamhaus.org
Neither of which return anything at all.
I would expect some sort of error message if the query was invalid in some way, but I just get nothing at all.
They may have concluded that you need to pay.
There is zero chance of me paying, judging them on the service thus far!
If you omit the +short you’ll probably see status: SERVFAIL in the reply.
If you switch to another resolver (e.g., OpenDNS) you might [EDIT: +see] things work:
admin@vm1:~$ dig +short @208.67.222.222 2.0.0.127.zen.spamhaus.org
127.0.0.4
127.0.0.10
127.0.0.2
I had this exact problem on a symbiosis bigv.io machine where all spamhaus.org lookups failed one day last December’ish, and forever more. Lookup volumes were low so I didn’t expect this to be a rate-limiting issue and talking to bytemark & spamhaus, I’m pretty sure it wasn’t due to my machine – yours might be.
Spamhaus were excellent throughout and pointed me at their Data Query Service.
Extracts from conversation with bytemark support will save me having to type 
…
I’ve been in touch with spamhaus-tech who couldn’t see any reason for the SERVFAILs (100% on my machine, or intermittent, as you reported). As a workaround, they’ve pointed me at their Data Query Service which is performing beautifully as we speak and offers a number of advantages over the public mirror lookups.
Leaders in IP and domain reputation data | Spamhaus Technology
The DQS provides everything that the public mirrors do and more, primarily, dynamic updates (updates within seconds of hailstorm attacks) and zero reputation domains. I expect a few false-positives with the latter but I’m keen to experiment. Even better, use of DQS is free, subject to the same conditions as the public mirrors.
DNSBL Usage Terms - The Spamhaus Project
There’s some ambiguity surrounding these terms - it may be deliberate - but when I contacted them about this is the past they basically gave me the nod. Incidentally, the enquiry sprung up as a result of discussion on the bytemark forum but now that’s been unceremoniously nuked I can’t update the dying symbiosis community, at source.
I had the impression - no more than that - that spamhaus would have said if it was know a known block (rate-limiting, etc) on their part. I’m sure they’re masters of networking dark arts and they certainly firewall abusive-looking hosts (often suspected to be barracuda appliances). However, it didn’t look like the case here but I’m no expert and it I didn’t want to press them.
1 Like
Many thanks for this.
Yes, leaving off the +short does indeed get me SERVFAIL in the responses. Completely different id: responses though.
I go to Sign up for a FREE Data Query Service Account - Spamhaus Technology but can’t get past the “Please enter your details:” line.
Seems that they are using something that my browser is not at all happy with. I tried several browsers with and without plugins and extensions. None worked.
[Later]
Managed it: using my phone, and 4G data, bypassing all the checks and protections, I managed to sign up for a 30 trial account. Not sure how I’ll use it, but it is step one of a journey.
I’ve augmented my github site with my email changes adding the ability to use a Spamhaus account with the Sympl (and probably the Symbiosis) exim4 setup in addition to the existing rules that use the public addresses. You need two files. This is tested and operational on my site. See
For the nftfw nftfwedit command just put KEY.zen.dq.spamhaus.net in the config.ini file in place of the existing Spamhaus line.
Incidentally, Spamhaus provide a SpamAssassin plugin that seems worthwhile and is a doddle to install.
1 Like
Just for info, I’ve been using it on symbiosis with a couple of files:
/etc/exim4/symbiosis.d/00-main/11-x-extra-macros
# define private query key used for spamhaus Data Query Service
# Sign up for a FREE Data Query Service Account - Spamhaus Technology
# Used in /etc/exim4/symbiosis.d/10-acl/50-acl-check-rcpt/74-x-dns-blacklists-spamhaus-data-query-service
SPAMHAUS_DQS_KEY = munge-my-private-key
… and …
/etc/exim4/symbiosis.d/10-acl/50-acl-check-rcpt/74-x-dns-blacklists-spamhaus-data-query-service
# Spamhaus Data Query Service (DQS)
# =================================
# Sign up for a FREE Data Query Service Account - Spamhaus Technology
#
# Use of the service requires an account and unique key
# The following criteria must be met for free-usage:
# DNSBL Usage Terms - The Spamhaus Project
#
# The key, SPAMHAUS_DQS_KEY, is set in /etc/exim4/symbiosis.d/00-main/11-x-extra-macros
#
# lookups depend on site-specific config files:
# /srv//config/blacklists/zen.dq.spamhaus.net : client ip
# /srv//config/blacklists/dbl.dq.spamhaus.net : domain (smtp reverse-path)
# /srv/*/config/blacklists/zrd.dq.spamhaus.net : zero-reputation-domains (reverse-path)
#
# zen subsets/combinations haven’t been used;
# - xbl.dq.spamhaus.net
# - pbl.dq.spamhaus.net
# - sbl-xbl.dq.spamhaus.net
#
#
# v20200212
# -
warn domains = +vhost_domains
# abort if the private-key macro hasn't been set
condition = ${if !eq {${length{15}{SPAMHAUS_DQS_KEY}}}{SPAMHAUS_DQS_KE}}
# populate dnslist where configured
set acl_m_dnslist1 = ${filter{ \
zen.dq.spamhaus.net!=127.255.255.252,127.255.255.254,127.255.255.255 : \
dbl.dq.spamhaus.net!=127.0.1.255,127.255.255.252,127.255.255.254,127.255.255.255/$sender_address_domain : \
zrd.dq.spamhaus.net!=127.255.255.252,127.255.255.254,127.255.255.255/$sender_address_domain \
} \
{exists{/srv/$domain/config/blacklists/${extract{1}{=!&/}{$item}{$value}{$item}}}} \
}
# abort if no services are configured
condition = ${if >{${listcount:$acl_m_dnslist1}}{0}}
# prepend the private key for the lookup hostnames [could have above, but clarity. Don't mention it ;)]
dnslists = ${map {$acl_m_dnslist1}{SPAMHAUS_DQS_KEY.$item}}
# Remove key & '.' from the first service to return a hit
# this is used for any rejection message & for logging (counts)
set acl_m_hit_by = ${substr{${strlen:SPAMHAUS_DQS_KEY.}}{$dnslist_domain}}
# set header in case we tag rather than reject
add_header = X-Spam-Blacklisted: $dnslist_matched listed at $acl_m_hit_by ($dnslist_value)
# define the action: if "tag"s not in the config file we reject
set acl_m105074_act = ${lookup{tag}lsearch{/srv/$domain/config/blacklists/$acl_m_hit_by}{tag}{reject}}
logwrite = ACL#10/50/74: H=$sender_fullhost $dnslist_matched blacklisted at $acl_m_hit_by ($dnslist_value) - $domain set to $acl_m105074_act
deny condition = ${if eq {$acl_m105074_act}{reject}}
message = $dnslist_matched blacklisted at $acl_m_hit_by ($dnslist_value) \
${if def:dnslist_text {\n$dnslist_text}}
[Hopefully, the contents survive the forum formatting]
What I did was to replace the entry in config/blacklists with one that says
99weweteeeheheh5ky.zen.dq.spamhaus.net (well, with my key, not that random text) and it seems to be working, I think. Not having the key there makes a lot more sense.
I’ve added the pcollinson suggestions, now so I’ll see how it goes.
I wondered if the mail should expose the key in a bounce message, which seems perhaps to be a bad idea… and the default settings for Sympl would have done that. Which I see alphacabbage1’s solution did worry about too.
Can you enlighten me on the clever bit…
zen.dq.spamhaus.net!=127.255.255.252,127.255.255.254,127.255.255.255
etc etc what’s happening here?
Arguably, it’s not clever as it hides misconfiguration errors. It’s using negation to say reject/tag if the response (isn’t nxdomain and) contains something other than the specified spamhaus error codes. Initially, I went that way as I didn’t want to risk rejecting legitimate mail because of dodgy syntax.
1 Like
I was wondering about doing some stuff for dbl and zrd - but it looks as if the SpamAssassin plugin will deal with that - not that I’ve had more than a cursory look.
Looking here, dbl is great, zrd hasn’t kicked in but is potentially very useful. The spambots have been quiet recently but this is what I’ve been seeing:
admin@vm1:~$ /srv/.all-sites/utils/rblinfo
23 rbl services currently configured
14 show rejections (exim4 logs with 10 day history)
non-spamhaus.org services might only 'tag'
service sites rejections
--------------------------------------------------------
zen.dq.spamhaus.net 32 183
b.barracudacentral.org 20 91
dbl.dq.spamhaus.net 32 38
truncate.gbudb.net 20 33
bl.mailspike.net 20 28
hostkarma.junkemailfilter.com 19 21
all.s5h.net 13 21
bl.spamcop.net 19 6
multi.uribl.com 19 4
dyna.spamrats.com 5 4
all.spamrats.com 12 3
ubl.unsubscore.com 10 2
noptr.spamrats.com 5 2
dnsbl.dronebl.org 16 1
zrd.dq.spamhaus.net 32 0
all.bl.blocklist.de 17 0
rhsbl.sorbs.net 12 0
dnsbl.justspam.org 6 0
bl.nordspam.com 3 0
dbl.nordspam.com 3 0
dbl.spamhaus.org 1 0
dnsbl.sorbs.net 1 0
zen.spamhaus.org 1 0
TOTAL - 437
--------------------------------------------------------
spamassassin 32 10
clamav 32 41
--------------------------------------------------------
debug3 oldest: /var/log/clamav/clamav.log.12.gz
DEBUG999 first line datetime string, maybe: Mon Apr 27 04:49:49 2020
DEBUG3.1 Glory 1587959389
DEBUG5 90
clamav logs (oldest file last modified 84 days ago)
- Sanesecurity.Spam - 111
- Sanesecurity.Junk - 43
- Sanesecurity.Phishing - 21
- Heuristics.Phishing - 7
- Sanesecurity.Jurlbl - 6
- Sanesecurity.Foxhole - 3
- Sanesecurity.SpamImg - 3
- Porcupine.Junk - 2
- Porcupine.Phishing - 2
- PhishTank.Phishing - 1
- Sanesecurity.Scam - 1
TOTAL - 200
--------------------------------------------------------
v20200214 : ~0.21s
(By the way, that spamhaus negation isn’t just about misconfiguration – it prevents all mail from being rejected if they started returning a code for, say, ‘too many queries’.)
“I’ve added the pcollinson suggestions, now so I’ll see how it goes.”
Well, it certainly seems to be working. However, the key is appearing in the log entries. Don’t suppose that matters.
Not sure what is in bounce messages
It’s in the log messages but not the bounce messages.
I seem to do very well with the old sendmail rule - reject if no reverse domain registered.
Yup, protocol violations still knock out loads of bad traffic. Some spam/uce operations are fully compliant though and churn through thousands of domains. Pre-GDPR days I had good mileage with greylisting and whois lookups. Specialist rbl like excommunicado An update on Communicado – hinterlands were particularly good. I suspect that spamhaus zrd (or similar) will have its day, here.