Zen.spamhaus.org timeouts

I’m getting hundreds or thousands of these lines in my logs each day (with different IP addresses, of course)

DNS list lookup defer (probably timeout) for 123.123.123.123.zen.spamhaus.org: assumed not in list: 1 Time(s)

What’s going wrong?

It looks like the server can’t do lookups against the Zen Spamhaus list.

Have a look at https://www.spamhaus.org/faq/section/DNSBL%20Usage#366 for info troubleshooting.

As far as I can make out, that is telling me to use
dig +short 80.68.80.24 1.0.0.127.zen.spamhaus.org
and
dig +short 80.68.80.24 2.0.0.127.zen.spamhaus.org
Neither of which return anything at all.

I would expect some sort of error message if the query was invalid in some way, but I just get nothing at all.

They may have concluded that you need to pay.

There is zero chance of me paying, judging them on the service thus far!

If you omit the +short you’ll probably see status: SERVFAIL in the reply.

If you switch to another resolver (e.g., OpenDNS) you might [EDIT: +see] things work:

admin@vm1:~$ dig +short @208.67.222.222 2.0.0.127.zen.spamhaus.org
127.0.0.4
127.0.0.10
127.0.0.2

I had this exact problem on a symbiosis bigv.io machine where all spamhaus.org lookups failed one day last December’ish, and forever more. Lookup volumes were low so I didn’t expect this to be a rate-limiting issue and talking to bytemark & spamhaus, I’m pretty sure it wasn’t due to my machine – yours might be.

Spamhaus were excellent throughout and pointed me at their Data Query Service.

Extracts from conversation with bytemark support will save me having to type :wink:

I’ve been in touch with spamhaus-tech who couldn’t see any reason for the SERVFAILs (100% on my machine, or intermittent, as you reported). As a workaround, they’ve pointed me at their Data Query Service which is performing beautifully as we speak and offers a number of advantages over the public mirror lookups.

https://www.spamhaustech.com/dqs/

The DQS provides everything that the public mirrors do and more, primarily, dynamic updates (updates within seconds of hailstorm attacks) and zero reputation domains. I expect a few false-positives with the latter but I’m keen to experiment. Even better, use of DQS is free, subject to the same conditions as the public mirrors.

https://www.spamhaus.org/organization/dnsblusage/

There’s some ambiguity surrounding these terms - it may be deliberate - but when I contacted them about this is the past they basically gave me the nod. Incidentally, the enquiry sprung up as a result of discussion on the bytemark forum but now that’s been unceremoniously nuked I can’t update the dying symbiosis community, at source. :wink:

I had the impression - no more than that - that spamhaus would have said if it was know a known block (rate-limiting, etc) on their part. I’m sure they’re masters of networking dark arts and they certainly firewall abusive-looking hosts (often suspected to be barracuda appliances). However, it didn’t look like the case here but I’m no expert and it I didn’t want to press them.

1 Like

Many thanks for this.

Yes, leaving off the +short does indeed get me SERVFAIL in the responses. Completely different id: responses though.

I go to https://www.spamhaustech.com/free-trial/sign-up-for-a-free-data-query-service-account/ but can’t get past the “Please enter your details:” line.
Seems that they are using something that my browser is not at all happy with. I tried several browsers with and without plugins and extensions. None worked.

[Later]
Managed it: using my phone, and 4G data, bypassing all the checks and protections, I managed to sign up for a 30 trial account. Not sure how I’ll use it, but it is step one of a journey.

I’ve augmented my github site with my email changes adding the ability to use a Spamhaus account with the Sympl (and probably the Symbiosis) exim4 setup in addition to the existing rules that use the public addresses. You need two files. This is tested and operational on my site. See

For the nftfw nftfwedit command just put KEY.zen.dq.spamhaus.net in the config.ini file in place of the existing Spamhaus line.

Incidentally, Spamhaus provide a SpamAssassin plugin that seems worthwhile and is a doddle to install.

1 Like

Just for info, I’ve been using it on symbiosis with a couple of files:

/etc/exim4/symbiosis.d/00-main/11-x-extra-macros
# define private query key used for spamhaus Data Query Service
# https://www.spamhaustech.com/dqs/

# Used in /etc/exim4/symbiosis.d/10-acl/50-acl-check-rcpt/74-x-dns-blacklists-spamhaus-data-query-service

SPAMHAUS_DQS_KEY = munge-my-private-key

… and …

/etc/exim4/symbiosis.d/10-acl/50-acl-check-rcpt/74-x-dns-blacklists-spamhaus-data-query-service
# Spamhaus Data Query Service (DQS)
# =================================
# https://www.spamhaustech.com/dqs/
#
# Use of the service requires an account and unique key
# The following criteria must be met for free-usage:
# https://www.spamhaus.org/organization/dnsblusage/
#
# The key, SPAMHAUS_DQS_KEY, is set in /etc/exim4/symbiosis.d/00-main/11-x-extra-macros
#
# lookups depend on site-specific config files:
# /srv//config/blacklists/zen.dq.spamhaus.net : client ip
# /srv/
/config/blacklists/dbl.dq.spamhaus.net : domain (smtp reverse-path)
# /srv/*/config/blacklists/zrd.dq.spamhaus.net : zero-reputation-domains (reverse-path)
#
# zen subsets/combinations haven’t been used;
# - xbl.dq.spamhaus.net
# - pbl.dq.spamhaus.net
# - sbl-xbl.dq.spamhaus.net
#
#
# v20200212
# -

    warn        domains             = +vhost_domains
                                      # abort if the private-key macro hasn't been set
                condition           = ${if !eq {${length{15}{SPAMHAUS_DQS_KEY}}}{SPAMHAUS_DQS_KE}}
                                      # populate dnslist where configured
                set acl_m_dnslist1  = ${filter{ \
                                                zen.dq.spamhaus.net!=127.255.255.252,127.255.255.254,127.255.255.255                                    : \
                                                dbl.dq.spamhaus.net!=127.0.1.255,127.255.255.252,127.255.255.254,127.255.255.255/$sender_address_domain : \
                                                zrd.dq.spamhaus.net!=127.255.255.252,127.255.255.254,127.255.255.255/$sender_address_domain               \
                                              } \
                                              {exists{/srv/$domain/config/blacklists/${extract{1}{=!&/}{$item}{$value}{$item}}}} \
                                      }
                                      # abort if no services are configured
                condition           = ${if >{${listcount:$acl_m_dnslist1}}{0}}
                                      # prepend the private key for the lookup hostnames [could have above, but clarity. Don't mention it ;)]
                dnslists            = ${map {$acl_m_dnslist1}{SPAMHAUS_DQS_KEY.$item}}
                                      # Remove key & '.' from the first service to return a hit
                                      # this is used for any rejection message & for logging (counts)
                set acl_m_hit_by    = ${substr{${strlen:SPAMHAUS_DQS_KEY.}}{$dnslist_domain}}
                                      # set header in case we tag rather than reject
                add_header          = X-Spam-Blacklisted: $dnslist_matched listed at $acl_m_hit_by ($dnslist_value)
                                      # define the action: if "tag"s not in the config file we reject
                set acl_m105074_act = ${lookup{tag}lsearch{/srv/$domain/config/blacklists/$acl_m_hit_by}{tag}{reject}}
                logwrite            = ACL#10/50/74: H=$sender_fullhost $dnslist_matched blacklisted at $acl_m_hit_by ($dnslist_value) - $domain set to $acl_m105074_act

    deny        condition           = ${if eq {$acl_m105074_act}{reject}}
                message             = $dnslist_matched blacklisted at $acl_m_hit_by ($dnslist_value) \
                                      ${if def:dnslist_text {\n$dnslist_text}}

[Hopefully, the contents survive the forum formatting]

What I did was to replace the entry in config/blacklists with one that says
99weweteeeheheh5ky.zen.dq.spamhaus.net (well, with my key, not that random text) and it seems to be working, I think. Not having the key there makes a lot more sense.
I’ve added the pcollinson suggestions, now so I’ll see how it goes.

I wondered if the mail should expose the key in a bounce message, which seems perhaps to be a bad idea… and the default settings for Sympl would have done that. Which I see alphacabbage1’s solution did worry about too.

Can you enlighten me on the clever bit…

zen.dq.spamhaus.net!=127.255.255.252,127.255.255.254,127.255.255.255 

etc etc what’s happening here?

Arguably, it’s not clever :wink: as it hides misconfiguration errors. It’s using negation to say reject/tag if the response (isn’t nxdomain and) contains something other than the specified spamhaus error codes. Initially, I went that way as I didn’t want to risk rejecting legitimate mail because of dodgy syntax.

1 Like

I was wondering about doing some stuff for dbl and zrd - but it looks as if the SpamAssassin plugin will deal with that - not that I’ve had more than a cursory look.

Looking here, dbl is great, zrd hasn’t kicked in but is potentially very useful. The spambots have been quiet recently but this is what I’ve been seeing:

admin@vm1:~$ /srv/.all-sites/utils/rblinfo
   23 rbl services currently configured
   14 show rejections (exim4 logs with 10 day history)
      non-spamhaus.org services might only 'tag'

  service                         sites     rejections
--------------------------------------------------------
  zen.dq.spamhaus.net                32            183
  b.barracudacentral.org             20             91
  dbl.dq.spamhaus.net                32             38
  truncate.gbudb.net                 20             33
  bl.mailspike.net                   20             28
  hostkarma.junkemailfilter.com      19             21
  all.s5h.net                        13             21
  bl.spamcop.net                     19              6
  multi.uribl.com                    19              4
  dyna.spamrats.com                   5              4
  all.spamrats.com                   12              3
  ubl.unsubscore.com                 10              2
  noptr.spamrats.com                  5              2
  dnsbl.dronebl.org                  16              1
  zrd.dq.spamhaus.net                32              0
  all.bl.blocklist.de                17              0
  rhsbl.sorbs.net                    12              0
  dnsbl.justspam.org                  6              0
  bl.nordspam.com                     3              0
  dbl.nordspam.com                    3              0
  dbl.spamhaus.org                    1              0
  dnsbl.sorbs.net                     1              0
  zen.spamhaus.org                    1              0
  TOTAL                               -            437
--------------------------------------------------------
  spamassassin                       32             10
  clamav                             32             41
--------------------------------------------------------
debug3 oldest: /var/log/clamav/clamav.log.12.gz
DEBUG999 first line datetime string, maybe: Mon Apr 27 04:49:49 2020
DEBUG3.1 Glory 1587959389
DEBUG5 90
  clamav logs (oldest file last modified 84 days ago)
  - Sanesecurity.Spam                 -            111
  - Sanesecurity.Junk                 -             43
  - Sanesecurity.Phishing             -             21
  - Heuristics.Phishing               -              7
  - Sanesecurity.Jurlbl               -              6
  - Sanesecurity.Foxhole              -              3
  - Sanesecurity.SpamImg              -              3
  - Porcupine.Junk                    -              2
  - Porcupine.Phishing                -              2
  - PhishTank.Phishing                -              1
  - Sanesecurity.Scam                 -              1
  TOTAL                               -            200
--------------------------------------------------------
  v20200214 : ~0.21s

(By the way, that spamhaus negation isn’t just about misconfiguration – it prevents all mail from being rejected if they started returning a code for, say, ‘too many queries’.)

“I’ve added the pcollinson suggestions, now so I’ll see how it goes.”
Well, it certainly seems to be working. However, the key is appearing in the log entries. Don’t suppose that matters.
Not sure what is in bounce messages

It’s in the log messages but not the bounce messages.

I seem to do very well with the old sendmail rule - reject if no reverse domain registered.

Yup, protocol violations still knock out loads of bad traffic. Some spam/uce operations are fully compliant though and churn through thousands of domains. Pre-GDPR days I had good mileage with greylisting and whois lookups. Specialist rbl like excommunicado https://blog.hinterlands.org/2013/11/an-update-on-communicado/ were particularly good. I suspect that spamhaus zrd (or similar) will have its day, here.