]> gitweb.michael.orlitzky.com - djbdns-logparse.git/log
djbdns-logparse.git
19 months agodoc/ChangeLog: update for v0.0.2. master 0.0.2
Michael Orlitzky [Fri, 23 Sep 2022 12:06:28 +0000 (08:06 -0400)]
doc/ChangeLog: update for v0.0.2.

19 months agosetup.py: bump to v0.0.2.
Michael Orlitzky [Fri, 23 Sep 2022 12:06:16 +0000 (08:06 -0400)]
setup.py: bump to v0.0.2.

19 months agodjbdns/io.py: fix test that fails in other timezones.
Michael Orlitzky [Fri, 23 Sep 2022 12:04:34 +0000 (08:04 -0400)]
djbdns/io.py: fix test that fails in other timezones.

19 months agodoc/*: tai64nlocal comes from daemontools, not djbdns.
Michael Orlitzky [Thu, 22 Sep 2022 19:56:06 +0000 (15:56 -0400)]
doc/*: tai64nlocal comes from daemontools, not djbdns.

19 months agodoc/ChangeLog: update for the pending v0.0.1 release. 0.0.1
Michael Orlitzky [Thu, 22 Sep 2022 14:17:42 +0000 (10:17 -0400)]
doc/ChangeLog: update for the pending v0.0.1 release.

19 months agodjbdns/*.py: add all remaining mappings to QUERY_TYPE_NAME.
Michael Orlitzky [Thu, 22 Sep 2022 14:13:47 +0000 (10:13 -0400)]
djbdns/*.py: add all remaining mappings to QUERY_TYPE_NAME.

The parser will now raise an exception if it encounters a type that
does not have an entry in the dictionary.

19 months agodjbdns/io.py: add a comment about an early return.
Michael Orlitzky [Thu, 22 Sep 2022 13:40:39 +0000 (09:40 -0400)]
djbdns/io.py: add a comment about an early return.

19 months agodjbdns/dnscache.py: de-indent a block that doesn't need to be guarded.
Michael Orlitzky [Thu, 22 Sep 2022 11:58:12 +0000 (07:58 -0400)]
djbdns/dnscache.py: de-indent a block that doesn't need to be guarded.

19 months agodjbdns/*.py: add module docstrings.
Michael Orlitzky [Thu, 22 Sep 2022 11:57:55 +0000 (07:57 -0400)]
djbdns/*.py: add module docstrings.

19 months agodjbdns/io.py: handle nonexistent Popen stdin/stdout.
Michael Orlitzky [Thu, 22 Sep 2022 02:18:18 +0000 (22:18 -0400)]
djbdns/io.py: handle nonexistent Popen stdin/stdout.

Type annotations say that this is possible, so... just quit if it
happens?

19 months agodjbdns/*.py: capitalize global variables.
Michael Orlitzky [Thu, 22 Sep 2022 01:49:38 +0000 (21:49 -0400)]
djbdns/*.py: capitalize global variables.

19 months agodjbdns/{dnscache,tinydns}.py: explicit imports from djbdns.common.
Michael Orlitzky [Thu, 22 Sep 2022 01:44:59 +0000 (21:44 -0400)]
djbdns/{dnscache,tinydns}.py: explicit imports from djbdns.common.

19 months agodjbdns/*.py: don't clobber id() and type() built-ins.
Michael Orlitzky [Thu, 22 Sep 2022 01:41:31 +0000 (21:41 -0400)]
djbdns/*.py: don't clobber id() and type() built-ins.

19 months agodjbdns/{dnscache,tinydns}.py: don't clobber the global compile().
Michael Orlitzky [Thu, 22 Sep 2022 01:30:35 +0000 (21:30 -0400)]
djbdns/{dnscache,tinydns}.py: don't clobber the global compile().

19 months agodjbdns/common.py: assume IPv6 if an address isn't IPv4.
Michael Orlitzky [Thu, 22 Sep 2022 01:25:59 +0000 (21:25 -0400)]
djbdns/common.py: assume IPv6 if an address isn't IPv4.

In other words, probably crash if convert_ip() is fed garbage. That's
OK. We could check the length of the hex string and raise a ValueError
if it isn't 8 or 32, but we might be solving problems that don't exist.

19 months agodjbdns/io.py: use Popen's context manager to clean up.
Michael Orlitzky [Thu, 22 Sep 2022 01:21:08 +0000 (21:21 -0400)]
djbdns/io.py: use Popen's context manager to clean up.

19 months agobin/djbdns-logparse: avoid clobbering exit() with sys.exit().
Michael Orlitzky [Thu, 22 Sep 2022 01:12:51 +0000 (21:12 -0400)]
bin/djbdns-logparse: avoid clobbering exit() with sys.exit().

19 months agodjbdns/dnscache.py: improve two comments.
Michael Orlitzky [Thu, 22 Sep 2022 01:00:15 +0000 (21:00 -0400)]
djbdns/dnscache.py: improve two comments.

19 months agodoc/man1/djbdns-logparse.1: document dnscache transformations.
Michael Orlitzky [Thu, 22 Sep 2022 00:56:57 +0000 (20:56 -0400)]
doc/man1/djbdns-logparse.1: document dnscache transformations.

19 months agodoc/man1/djbdns-logparse.1: add tinydns-by-filename example.
Michael Orlitzky [Tue, 20 Sep 2022 23:52:06 +0000 (19:52 -0400)]
doc/man1/djbdns-logparse.1: add tinydns-by-filename example.

19 months agodjbdns/io.py: handle BrokenPipeError in parse_logfile().
Michael Orlitzky [Tue, 20 Sep 2022 23:48:10 +0000 (19:48 -0400)]
djbdns/io.py: handle BrokenPipeError in parse_logfile().

This avoids a (harmless) crash when the output of djbdns-logparse is
piped to a process (such as head/tail) that closes the pipe before
djbdns-logparse is done writing to it.

19 months agobin/djbdns-logparse: handle SIGINT (again).
Michael Orlitzky [Tue, 20 Sep 2022 23:21:28 +0000 (19:21 -0400)]
bin/djbdns-logparse: handle SIGINT (again).

I removed the KeyboardInterrupt handler from the original program and
have now put it back as an async handler for SIGINT. This is helpful
when the user accidentally starts the program with no arguments and it
just sits there until he hits Ctrl-C (which would previously have
shown a stack trace).

19 months agodoc/man1/djbdns-logparse.1: finish the tinydns transformation docs.
Michael Orlitzky [Tue, 20 Sep 2022 01:12:50 +0000 (21:12 -0400)]
doc/man1/djbdns-logparse.1: finish the tinydns transformation docs.

19 months agosetup.py,doc/*: shorten the programs description by a few characters.
Michael Orlitzky [Tue, 20 Sep 2022 00:34:12 +0000 (20:34 -0400)]
setup.py,doc/*: shorten the programs description by a few characters.

19 months agodoc/man1/djbdns-logparse.1: utilize some fancy groff_man macros.
Michael Orlitzky [Tue, 20 Sep 2022 00:32:58 +0000 (20:32 -0400)]
doc/man1/djbdns-logparse.1: utilize some fancy groff_man macros.

19 months agodoc/man1/djbdns-logparse.1: start a man page.
Michael Orlitzky [Mon, 19 Sep 2022 15:08:54 +0000 (11:08 -0400)]
doc/man1/djbdns-logparse.1: start a man page.

19 months agodjbdns/dnscache.py: add docstring for decode_type().
Michael Orlitzky [Sat, 17 Sep 2022 23:51:34 +0000 (19:51 -0400)]
djbdns/dnscache.py: add docstring for decode_type().

19 months agodoc/README: mention usage and tests.
Michael Orlitzky [Sat, 17 Sep 2022 11:19:01 +0000 (07:19 -0400)]
doc/README: mention usage and tests.

19 months ago*/*: reorganize for easy testing using setuptools or tox.
Michael Orlitzky [Sat, 17 Sep 2022 00:07:05 +0000 (20:07 -0400)]
*/*: reorganize for easy testing using setuptools or tox.

Now running either,

  $ python setup.py test

or

  $ tox

will execute the test suite. (The former is deprecated, but is easier
for now, and most importantly doesn't require tox.)

19 months agosetup.py: add keywords.
Michael Orlitzky [Fri, 16 Sep 2022 23:32:00 +0000 (19:32 -0400)]
setup.py: add keywords.

19 months ago*/*: refactor into a driver program and a library.
Michael Orlitzky [Fri, 16 Sep 2022 23:16:57 +0000 (19:16 -0400)]
*/*: refactor into a driver program and a library.

Now there are a few separate modules in a "djbdns" package, and the
djbdns-logparse executable that makes use of them contains only what
the main() function used to.

19 months agobin/djbdns-logparse.py: add a dosctring for decode_serial().
Michael Orlitzky [Fri, 16 Sep 2022 19:18:46 +0000 (15:18 -0400)]
bin/djbdns-logparse.py: add a dosctring for decode_serial().

19 months agobin/djbdns-logparse.py: improve existing decode_* docs.
Michael Orlitzky [Fri, 16 Sep 2022 19:13:39 +0000 (15:13 -0400)]
bin/djbdns-logparse.py: improve existing decode_* docs.

19 months agobin/djbdns-logparse.py: add a docstring for decode_ttl().
Michael Orlitzky [Fri, 16 Sep 2022 18:57:32 +0000 (14:57 -0400)]
bin/djbdns-logparse.py: add a docstring for decode_ttl().

19 months agobin/djbdns-logparse.py: make all docstrings raw to avoid surprises.
Michael Orlitzky [Fri, 16 Sep 2022 18:46:26 +0000 (14:46 -0400)]
bin/djbdns-logparse.py: make all docstrings raw to avoid surprises.

19 months agobin/djbdns-logparse.py: add a few more type hints where forgotten.
Michael Orlitzky [Fri, 16 Sep 2022 18:45:24 +0000 (14:45 -0400)]
bin/djbdns-logparse.py: add a few more type hints where forgotten.

19 months agobin/djbdns-logparse.py: add docstring for decode_ip().
Michael Orlitzky [Fri, 16 Sep 2022 18:44:15 +0000 (14:44 -0400)]
bin/djbdns-logparse.py: add docstring for decode_ip().

19 months agobin/djbdns-logparse.py: use a variable to simplify a string template.
Michael Orlitzky [Fri, 16 Sep 2022 00:26:32 +0000 (20:26 -0400)]
bin/djbdns-logparse.py: use a variable to simplify a string template.

19 months agobin/djbdns-logparse.py: remove unused time imports.
Michael Orlitzky [Fri, 16 Sep 2022 00:23:49 +0000 (20:23 -0400)]
bin/djbdns-logparse.py: remove unused time imports.

19 months agobin/djbdns-logparse.py: localize subprocess imports.
Michael Orlitzky [Fri, 16 Sep 2022 00:22:46 +0000 (20:22 -0400)]
bin/djbdns-logparse.py: localize subprocess imports.

19 months agobin/djbdns-logparse.py: more docs for convert_ip().
Michael Orlitzky [Fri, 16 Sep 2022 00:20:19 +0000 (20:20 -0400)]
bin/djbdns-logparse.py: more docs for convert_ip().

19 months agobin/djbdns-logparse.py: add docstring for decode_client().
Michael Orlitzky [Fri, 16 Sep 2022 00:13:17 +0000 (20:13 -0400)]
bin/djbdns-logparse.py: add docstring for decode_client().

19 months agobin/djbdns-logparse.py: use f-strings for all string interpolation.
Michael Orlitzky [Thu, 15 Sep 2022 23:49:49 +0000 (19:49 -0400)]
bin/djbdns-logparse.py: use f-strings for all string interpolation.

19 months agobin/djbdns-logparse.py: fix python3 integer division for txt records.
Michael Orlitzky [Thu, 15 Sep 2022 23:49:21 +0000 (19:49 -0400)]
bin/djbdns-logparse.py: fix python3 integer division for txt records.

19 months agobin/djbdns-logparse.py: refactor handle_* types again.
Michael Orlitzky [Thu, 15 Sep 2022 23:18:01 +0000 (19:18 -0400)]
bin/djbdns-logparse.py: refactor handle_* types again.

Instead of using True/False to signal success after possibly using
print() to actually write the result, we not just return either the
processed string, or None.

19 months agobin/djbdns-logparse.py: add docstring for handle_dnscache_log().
Michael Orlitzky [Thu, 15 Sep 2022 23:02:22 +0000 (19:02 -0400)]
bin/djbdns-logparse.py: add docstring for handle_dnscache_log().

19 months agobin/djbdns-logparse.py: more doctests for handle_tinydns_log().
Michael Orlitzky [Thu, 15 Sep 2022 23:01:16 +0000 (19:01 -0400)]
bin/djbdns-logparse.py: more doctests for handle_tinydns_log().

19 months agobin/djbdns-logparse.py: move regex handling down a level.
Michael Orlitzky [Thu, 15 Sep 2022 22:50:26 +0000 (18:50 -0400)]
bin/djbdns-logparse.py: move regex handling down a level.

It was a bit awkward to match a line against a regex, and then to have
to immediately pass that match object into a function that knows what
to do with it (to avoid duplicating work). Now the handle_tinydns_log()
and handle_dnscache_log() functions create the match object themselves,
and signal success or failure by returning a boolean.

19 months agobin/djbdns-logparse.py: add a docstring for main().
Michael Orlitzky [Thu, 15 Sep 2022 15:23:00 +0000 (11:23 -0400)]
bin/djbdns-logparse.py: add a docstring for main().

19 months agobin/djbdns-logparse.py: rework some of the tinydns logic.
Michael Orlitzky [Thu, 15 Sep 2022 15:20:36 +0000 (11:20 -0400)]
bin/djbdns-logparse.py: rework some of the tinydns logic.

19 months agobin/djbdns-logparse.py: add doctest for parse_logfile().
Michael Orlitzky [Thu, 15 Sep 2022 14:15:46 +0000 (10:15 -0400)]
bin/djbdns-logparse.py: add doctest for parse_logfile().

19 months agobin/djbdns-logparse.py: add comment about open file descriptors.
Michael Orlitzky [Thu, 15 Sep 2022 14:01:07 +0000 (10:01 -0400)]
bin/djbdns-logparse.py: add comment about open file descriptors.

19 months agobin/djbdns-logparse.py: add docs for parse_logfile().
Michael Orlitzky [Thu, 15 Sep 2022 13:46:16 +0000 (09:46 -0400)]
bin/djbdns-logparse.py: add docs for parse_logfile().

19 months agobin/djbdns-logparse.py: drop unused _cvt_ip()/_cvt_port() functions.
Michael Orlitzky [Thu, 15 Sep 2022 13:31:27 +0000 (09:31 -0400)]
bin/djbdns-logparse.py: drop unused _cvt_ip()/_cvt_port() functions.

19 months agobin/djbdns-logparse.py: use f-strings for regex patterns; add comments.
Michael Orlitzky [Thu, 15 Sep 2022 13:30:18 +0000 (09:30 -0400)]
bin/djbdns-logparse.py: use f-strings for regex patterns; add comments.

19 months agobin/djbdns-logparse.py: add docs/examples for handle_tinydns_log().
Michael Orlitzky [Thu, 15 Sep 2022 03:10:48 +0000 (23:10 -0400)]
bin/djbdns-logparse.py: add docs/examples for handle_tinydns_log().

19 months agobin/djbdns-logparse.py: fiddle with the convert_ip() comments.
Michael Orlitzky [Thu, 15 Sep 2022 03:10:17 +0000 (23:10 -0400)]
bin/djbdns-logparse.py: fiddle with the convert_ip() comments.

19 months agobin/djbdns-logparse.py: add some newer type mappings.
Michael Orlitzky [Thu, 15 Sep 2022 02:53:37 +0000 (22:53 -0400)]
bin/djbdns-logparse.py: add some newer type mappings.

I've added srv, naptr, dnskey, tlsa, https, and caa type mappings
based on what my own logs contain. Only SRV is actually supported by
tinydns. There are probably others in use today that will show up
eventually.

19 months agobin/djbdns-logparse.py: add .py extension and doctests for convert_ip().
Michael Orlitzky [Thu, 15 Sep 2022 01:30:53 +0000 (21:30 -0400)]
bin/djbdns-logparse.py: add .py extension and doctests for convert_ip().

Without the rename, "python -m doctest" isn't smart enough to figure
out that this thing is a python file.

19 months agodoc/README: siphon the description and history from bin/djbdns-logparse.
Michael Orlitzky [Thu, 15 Sep 2022 01:20:47 +0000 (21:20 -0400)]
doc/README: siphon the description and history from bin/djbdns-logparse.

19 months agobin/djbdns-logparse: only import sys.stdin within main().
Michael Orlitzky [Thu, 15 Sep 2022 00:54:35 +0000 (20:54 -0400)]
bin/djbdns-logparse: only import sys.stdin within main().

19 months agobin/djbdns-logparse: remove unused warn() function.
Michael Orlitzky [Thu, 15 Sep 2022 00:54:17 +0000 (20:54 -0400)]
bin/djbdns-logparse: remove unused warn() function.

19 months agobin/djbdns-logparse: fix line splitting in tinydns output.
Michael Orlitzky [Thu, 15 Sep 2022 00:51:00 +0000 (20:51 -0400)]
bin/djbdns-logparse: fix line splitting in tinydns output.

19 months agobin/djbdns-logparse: use argparse for arg parse.
Michael Orlitzky [Thu, 15 Sep 2022 00:44:09 +0000 (20:44 -0400)]
bin/djbdns-logparse: use argparse for arg parse.

19 months agoInitial commit, just starting to organize everything.
Michael Orlitzky [Thu, 15 Sep 2022 00:11:03 +0000 (20:11 -0400)]
Initial commit, just starting to organize everything.