]> gitweb.michael.orlitzky.com - djbdns-logparse.git/log
djbdns-logparse.git
2 years agodjbdns/io.py: fix test that fails in other timezones.
Michael Orlitzky [Fri, 23 Sep 2022 12:04:34 +0000 (08:04 -0400)]
djbdns/io.py: fix test that fails in other timezones.

2 years agodoc/*: tai64nlocal comes from daemontools, not djbdns.
Michael Orlitzky [Thu, 22 Sep 2022 19:56:06 +0000 (15:56 -0400)]
doc/*: tai64nlocal comes from daemontools, not djbdns.

2 years agodoc/ChangeLog: update for the pending v0.0.1 release. 0.0.1
Michael Orlitzky [Thu, 22 Sep 2022 14:17:42 +0000 (10:17 -0400)]
doc/ChangeLog: update for the pending v0.0.1 release.

2 years agodjbdns/*.py: add all remaining mappings to QUERY_TYPE_NAME.
Michael Orlitzky [Thu, 22 Sep 2022 14:13:47 +0000 (10:13 -0400)]
djbdns/*.py: add all remaining mappings to QUERY_TYPE_NAME.

The parser will now raise an exception if it encounters a type that
does not have an entry in the dictionary.

2 years agodjbdns/io.py: add a comment about an early return.
Michael Orlitzky [Thu, 22 Sep 2022 13:40:39 +0000 (09:40 -0400)]
djbdns/io.py: add a comment about an early return.

2 years agodjbdns/dnscache.py: de-indent a block that doesn't need to be guarded.
Michael Orlitzky [Thu, 22 Sep 2022 11:58:12 +0000 (07:58 -0400)]
djbdns/dnscache.py: de-indent a block that doesn't need to be guarded.

2 years agodjbdns/*.py: add module docstrings.
Michael Orlitzky [Thu, 22 Sep 2022 11:57:55 +0000 (07:57 -0400)]
djbdns/*.py: add module docstrings.

2 years agodjbdns/io.py: handle nonexistent Popen stdin/stdout.
Michael Orlitzky [Thu, 22 Sep 2022 02:18:18 +0000 (22:18 -0400)]
djbdns/io.py: handle nonexistent Popen stdin/stdout.

Type annotations say that this is possible, so... just quit if it
happens?

2 years agodjbdns/*.py: capitalize global variables.
Michael Orlitzky [Thu, 22 Sep 2022 01:49:38 +0000 (21:49 -0400)]
djbdns/*.py: capitalize global variables.

2 years agodjbdns/{dnscache,tinydns}.py: explicit imports from djbdns.common.
Michael Orlitzky [Thu, 22 Sep 2022 01:44:59 +0000 (21:44 -0400)]
djbdns/{dnscache,tinydns}.py: explicit imports from djbdns.common.

2 years agodjbdns/*.py: don't clobber id() and type() built-ins.
Michael Orlitzky [Thu, 22 Sep 2022 01:41:31 +0000 (21:41 -0400)]
djbdns/*.py: don't clobber id() and type() built-ins.

2 years agodjbdns/{dnscache,tinydns}.py: don't clobber the global compile().
Michael Orlitzky [Thu, 22 Sep 2022 01:30:35 +0000 (21:30 -0400)]
djbdns/{dnscache,tinydns}.py: don't clobber the global compile().

2 years agodjbdns/common.py: assume IPv6 if an address isn't IPv4.
Michael Orlitzky [Thu, 22 Sep 2022 01:25:59 +0000 (21:25 -0400)]
djbdns/common.py: assume IPv6 if an address isn't IPv4.

In other words, probably crash if convert_ip() is fed garbage. That's
OK. We could check the length of the hex string and raise a ValueError
if it isn't 8 or 32, but we might be solving problems that don't exist.

2 years agodjbdns/io.py: use Popen's context manager to clean up.
Michael Orlitzky [Thu, 22 Sep 2022 01:21:08 +0000 (21:21 -0400)]
djbdns/io.py: use Popen's context manager to clean up.

2 years agobin/djbdns-logparse: avoid clobbering exit() with sys.exit().
Michael Orlitzky [Thu, 22 Sep 2022 01:12:51 +0000 (21:12 -0400)]
bin/djbdns-logparse: avoid clobbering exit() with sys.exit().

2 years agodjbdns/dnscache.py: improve two comments.
Michael Orlitzky [Thu, 22 Sep 2022 01:00:15 +0000 (21:00 -0400)]
djbdns/dnscache.py: improve two comments.

2 years agodoc/man1/djbdns-logparse.1: document dnscache transformations.
Michael Orlitzky [Thu, 22 Sep 2022 00:56:57 +0000 (20:56 -0400)]
doc/man1/djbdns-logparse.1: document dnscache transformations.

2 years agodoc/man1/djbdns-logparse.1: add tinydns-by-filename example.
Michael Orlitzky [Tue, 20 Sep 2022 23:52:06 +0000 (19:52 -0400)]
doc/man1/djbdns-logparse.1: add tinydns-by-filename example.

2 years agodjbdns/io.py: handle BrokenPipeError in parse_logfile().
Michael Orlitzky [Tue, 20 Sep 2022 23:48:10 +0000 (19:48 -0400)]
djbdns/io.py: handle BrokenPipeError in parse_logfile().

This avoids a (harmless) crash when the output of djbdns-logparse is
piped to a process (such as head/tail) that closes the pipe before
djbdns-logparse is done writing to it.

2 years agobin/djbdns-logparse: handle SIGINT (again).
Michael Orlitzky [Tue, 20 Sep 2022 23:21:28 +0000 (19:21 -0400)]
bin/djbdns-logparse: handle SIGINT (again).

I removed the KeyboardInterrupt handler from the original program and
have now put it back as an async handler for SIGINT. This is helpful
when the user accidentally starts the program with no arguments and it
just sits there until he hits Ctrl-C (which would previously have
shown a stack trace).

2 years agodoc/man1/djbdns-logparse.1: finish the tinydns transformation docs.
Michael Orlitzky [Tue, 20 Sep 2022 01:12:50 +0000 (21:12 -0400)]
doc/man1/djbdns-logparse.1: finish the tinydns transformation docs.

2 years agosetup.py,doc/*: shorten the programs description by a few characters.
Michael Orlitzky [Tue, 20 Sep 2022 00:34:12 +0000 (20:34 -0400)]
setup.py,doc/*: shorten the programs description by a few characters.

2 years agodoc/man1/djbdns-logparse.1: utilize some fancy groff_man macros.
Michael Orlitzky [Tue, 20 Sep 2022 00:32:58 +0000 (20:32 -0400)]
doc/man1/djbdns-logparse.1: utilize some fancy groff_man macros.

2 years agodoc/man1/djbdns-logparse.1: start a man page.
Michael Orlitzky [Mon, 19 Sep 2022 15:08:54 +0000 (11:08 -0400)]
doc/man1/djbdns-logparse.1: start a man page.

2 years agodjbdns/dnscache.py: add docstring for decode_type().
Michael Orlitzky [Sat, 17 Sep 2022 23:51:34 +0000 (19:51 -0400)]
djbdns/dnscache.py: add docstring for decode_type().

2 years agodoc/README: mention usage and tests.
Michael Orlitzky [Sat, 17 Sep 2022 11:19:01 +0000 (07:19 -0400)]
doc/README: mention usage and tests.

2 years ago*/*: reorganize for easy testing using setuptools or tox.
Michael Orlitzky [Sat, 17 Sep 2022 00:07:05 +0000 (20:07 -0400)]
*/*: reorganize for easy testing using setuptools or tox.

Now running either,

  $ python setup.py test

or

  $ tox

will execute the test suite. (The former is deprecated, but is easier
for now, and most importantly doesn't require tox.)

2 years agosetup.py: add keywords.
Michael Orlitzky [Fri, 16 Sep 2022 23:32:00 +0000 (19:32 -0400)]
setup.py: add keywords.

2 years ago*/*: refactor into a driver program and a library.
Michael Orlitzky [Fri, 16 Sep 2022 23:16:57 +0000 (19:16 -0400)]
*/*: refactor into a driver program and a library.

Now there are a few separate modules in a "djbdns" package, and the
djbdns-logparse executable that makes use of them contains only what
the main() function used to.

2 years agobin/djbdns-logparse.py: add a dosctring for decode_serial().
Michael Orlitzky [Fri, 16 Sep 2022 19:18:46 +0000 (15:18 -0400)]
bin/djbdns-logparse.py: add a dosctring for decode_serial().

2 years agobin/djbdns-logparse.py: improve existing decode_* docs.
Michael Orlitzky [Fri, 16 Sep 2022 19:13:39 +0000 (15:13 -0400)]
bin/djbdns-logparse.py: improve existing decode_* docs.

2 years agobin/djbdns-logparse.py: add a docstring for decode_ttl().
Michael Orlitzky [Fri, 16 Sep 2022 18:57:32 +0000 (14:57 -0400)]
bin/djbdns-logparse.py: add a docstring for decode_ttl().

2 years agobin/djbdns-logparse.py: make all docstrings raw to avoid surprises.
Michael Orlitzky [Fri, 16 Sep 2022 18:46:26 +0000 (14:46 -0400)]
bin/djbdns-logparse.py: make all docstrings raw to avoid surprises.

2 years agobin/djbdns-logparse.py: add a few more type hints where forgotten.
Michael Orlitzky [Fri, 16 Sep 2022 18:45:24 +0000 (14:45 -0400)]
bin/djbdns-logparse.py: add a few more type hints where forgotten.

2 years agobin/djbdns-logparse.py: add docstring for decode_ip().
Michael Orlitzky [Fri, 16 Sep 2022 18:44:15 +0000 (14:44 -0400)]
bin/djbdns-logparse.py: add docstring for decode_ip().

2 years agobin/djbdns-logparse.py: use a variable to simplify a string template.
Michael Orlitzky [Fri, 16 Sep 2022 00:26:32 +0000 (20:26 -0400)]
bin/djbdns-logparse.py: use a variable to simplify a string template.

2 years agobin/djbdns-logparse.py: remove unused time imports.
Michael Orlitzky [Fri, 16 Sep 2022 00:23:49 +0000 (20:23 -0400)]
bin/djbdns-logparse.py: remove unused time imports.

2 years agobin/djbdns-logparse.py: localize subprocess imports.
Michael Orlitzky [Fri, 16 Sep 2022 00:22:46 +0000 (20:22 -0400)]
bin/djbdns-logparse.py: localize subprocess imports.

2 years agobin/djbdns-logparse.py: more docs for convert_ip().
Michael Orlitzky [Fri, 16 Sep 2022 00:20:19 +0000 (20:20 -0400)]
bin/djbdns-logparse.py: more docs for convert_ip().

2 years agobin/djbdns-logparse.py: add docstring for decode_client().
Michael Orlitzky [Fri, 16 Sep 2022 00:13:17 +0000 (20:13 -0400)]
bin/djbdns-logparse.py: add docstring for decode_client().

2 years agobin/djbdns-logparse.py: use f-strings for all string interpolation.
Michael Orlitzky [Thu, 15 Sep 2022 23:49:49 +0000 (19:49 -0400)]
bin/djbdns-logparse.py: use f-strings for all string interpolation.

2 years agobin/djbdns-logparse.py: fix python3 integer division for txt records.
Michael Orlitzky [Thu, 15 Sep 2022 23:49:21 +0000 (19:49 -0400)]
bin/djbdns-logparse.py: fix python3 integer division for txt records.

2 years agobin/djbdns-logparse.py: refactor handle_* types again.
Michael Orlitzky [Thu, 15 Sep 2022 23:18:01 +0000 (19:18 -0400)]
bin/djbdns-logparse.py: refactor handle_* types again.

Instead of using True/False to signal success after possibly using
print() to actually write the result, we not just return either the
processed string, or None.

2 years agobin/djbdns-logparse.py: add docstring for handle_dnscache_log().
Michael Orlitzky [Thu, 15 Sep 2022 23:02:22 +0000 (19:02 -0400)]
bin/djbdns-logparse.py: add docstring for handle_dnscache_log().

2 years agobin/djbdns-logparse.py: more doctests for handle_tinydns_log().
Michael Orlitzky [Thu, 15 Sep 2022 23:01:16 +0000 (19:01 -0400)]
bin/djbdns-logparse.py: more doctests for handle_tinydns_log().

2 years agobin/djbdns-logparse.py: move regex handling down a level.
Michael Orlitzky [Thu, 15 Sep 2022 22:50:26 +0000 (18:50 -0400)]
bin/djbdns-logparse.py: move regex handling down a level.

It was a bit awkward to match a line against a regex, and then to have
to immediately pass that match object into a function that knows what
to do with it (to avoid duplicating work). Now the handle_tinydns_log()
and handle_dnscache_log() functions create the match object themselves,
and signal success or failure by returning a boolean.

2 years agobin/djbdns-logparse.py: add a docstring for main().
Michael Orlitzky [Thu, 15 Sep 2022 15:23:00 +0000 (11:23 -0400)]
bin/djbdns-logparse.py: add a docstring for main().

2 years agobin/djbdns-logparse.py: rework some of the tinydns logic.
Michael Orlitzky [Thu, 15 Sep 2022 15:20:36 +0000 (11:20 -0400)]
bin/djbdns-logparse.py: rework some of the tinydns logic.

2 years agobin/djbdns-logparse.py: add doctest for parse_logfile().
Michael Orlitzky [Thu, 15 Sep 2022 14:15:46 +0000 (10:15 -0400)]
bin/djbdns-logparse.py: add doctest for parse_logfile().

2 years agobin/djbdns-logparse.py: add comment about open file descriptors.
Michael Orlitzky [Thu, 15 Sep 2022 14:01:07 +0000 (10:01 -0400)]
bin/djbdns-logparse.py: add comment about open file descriptors.

2 years agobin/djbdns-logparse.py: add docs for parse_logfile().
Michael Orlitzky [Thu, 15 Sep 2022 13:46:16 +0000 (09:46 -0400)]
bin/djbdns-logparse.py: add docs for parse_logfile().

2 years agobin/djbdns-logparse.py: drop unused _cvt_ip()/_cvt_port() functions.
Michael Orlitzky [Thu, 15 Sep 2022 13:31:27 +0000 (09:31 -0400)]
bin/djbdns-logparse.py: drop unused _cvt_ip()/_cvt_port() functions.

2 years agobin/djbdns-logparse.py: use f-strings for regex patterns; add comments.
Michael Orlitzky [Thu, 15 Sep 2022 13:30:18 +0000 (09:30 -0400)]
bin/djbdns-logparse.py: use f-strings for regex patterns; add comments.

2 years agobin/djbdns-logparse.py: add docs/examples for handle_tinydns_log().
Michael Orlitzky [Thu, 15 Sep 2022 03:10:48 +0000 (23:10 -0400)]
bin/djbdns-logparse.py: add docs/examples for handle_tinydns_log().

2 years agobin/djbdns-logparse.py: fiddle with the convert_ip() comments.
Michael Orlitzky [Thu, 15 Sep 2022 03:10:17 +0000 (23:10 -0400)]
bin/djbdns-logparse.py: fiddle with the convert_ip() comments.

2 years agobin/djbdns-logparse.py: add some newer type mappings.
Michael Orlitzky [Thu, 15 Sep 2022 02:53:37 +0000 (22:53 -0400)]
bin/djbdns-logparse.py: add some newer type mappings.

I've added srv, naptr, dnskey, tlsa, https, and caa type mappings
based on what my own logs contain. Only SRV is actually supported by
tinydns. There are probably others in use today that will show up
eventually.

2 years agobin/djbdns-logparse.py: add .py extension and doctests for convert_ip().
Michael Orlitzky [Thu, 15 Sep 2022 01:30:53 +0000 (21:30 -0400)]
bin/djbdns-logparse.py: add .py extension and doctests for convert_ip().

Without the rename, "python -m doctest" isn't smart enough to figure
out that this thing is a python file.

2 years agodoc/README: siphon the description and history from bin/djbdns-logparse.
Michael Orlitzky [Thu, 15 Sep 2022 01:20:47 +0000 (21:20 -0400)]
doc/README: siphon the description and history from bin/djbdns-logparse.

2 years agobin/djbdns-logparse: only import sys.stdin within main().
Michael Orlitzky [Thu, 15 Sep 2022 00:54:35 +0000 (20:54 -0400)]
bin/djbdns-logparse: only import sys.stdin within main().

2 years agobin/djbdns-logparse: remove unused warn() function.
Michael Orlitzky [Thu, 15 Sep 2022 00:54:17 +0000 (20:54 -0400)]
bin/djbdns-logparse: remove unused warn() function.

2 years agobin/djbdns-logparse: fix line splitting in tinydns output.
Michael Orlitzky [Thu, 15 Sep 2022 00:51:00 +0000 (20:51 -0400)]
bin/djbdns-logparse: fix line splitting in tinydns output.

2 years agobin/djbdns-logparse: use argparse for arg parse.
Michael Orlitzky [Thu, 15 Sep 2022 00:44:09 +0000 (20:44 -0400)]
bin/djbdns-logparse: use argparse for arg parse.

2 years agoInitial commit, just starting to organize everything.
Michael Orlitzky [Thu, 15 Sep 2022 00:11:03 +0000 (20:11 -0400)]
Initial commit, just starting to organize everything.