4 author: Michael Orlitzky
5 maintainer: Michael Orlitzky <michael@orlitzky.com>
8 license-file: doc/LICENSE
12 doc/htsn-importrc.example
13 doc/man1/htsn-import.1
15 doc/README.development
20 schemagen/AutoRacingResultsXML/*.xml
21 schemagen/Auto_Racing_Schedule_XML/*.xml
22 schemagen/Heartbeat/*.xml
23 schemagen/injuriesxml/*.xml
24 schemagen/Injuries_Detail_XML/*.xml
25 schemagen/newsxml/*.xml
26 schemagen/Odds_XML/*.xml
27 schemagen/scoresxml/*.xml
28 schemagen/weatherxml/*.xml
33 Import XML files from The Sports Network into an RDBMS.
38 htsn-import [OPTIONS] [FILES]
41 The Sports Network <http://www.sportsnetwork.com/> offers an XML feed
42 containing various sports news and statistics. Our sister program
43 /htsn/ is capable of retrieving the feed and saving the individual
44 XML documents contained therein. But what to do with them?
46 The purpose of /htsn-import/ is to take these XML documents and
47 get them into something we can use, a relational database management
48 system (RDBMS), loosely known as a SQL database. The structure of
49 relational database, is, well, relational, and the feed XML is not. So
50 there is some work to do before the data can be inserted.
52 First, we must parse the XML. Each supported document type (see below)
53 has a full pickle/unpickle implementation (\"pickle\" is simply a
54 synonym for serialize here). That means that we parse the entire
55 document into a data structure, and if we pickle (serialize) that data
56 structure, we get the exact same XML document tha we started with.
58 This is important for two reasons. First, it serves as a second level
59 of validation. The first validation is performed by the XML parser,
60 but if that succeeds and unpicking fails, we know that something is
61 fishy. Second, we don't ever want to be surprised by some new element
62 or attribute showing up in the XML. The fact that we can unpickle the
63 whole thing now means that we won't be surprised in the future.
65 The aforementioned feature is especially important because we
66 automatically migrate the database schema every time we import a
67 document. If you attempt to import a \"newsxml.dtd\" document, all
68 database objects relating to the news will be created if they do not
69 exist. We don't want the schema to change out from under us without
70 warning, so it's important that no XML be parsed that would result in
71 a different schema than we had previously. Since we can
72 pickle/unpickle everything already, this should be impossible.
74 Examples and usage documentation are available in the man page.
76 executable htsn-import
87 groundhog-postgresql >= 0.5,
88 groundhog-sqlite >= 0.5,
111 OptionalConfiguration
118 TSN.XML.AutoRacingResults
119 TSN.XML.AutoRacingSchedule
123 TSN.XML.InjuriesDetail
135 -fwarn-missing-signatures
136 -fwarn-name-shadowing
140 -fwarn-incomplete-record-updates
141 -fwarn-monomorphism-restriction
142 -fwarn-unused-do-bind
149 -- The following unbreak profiling with template haskell. We have
150 -- to build the program twice; once without profile and again with
157 type: exitcode-stdio-1.0
158 hs-source-dirs: src test
159 main-is: TestSuite.hs
167 htsn-common >= 0.0.1,
170 groundhog-postgresql >= 0.5,
171 groundhog-sqlite >= 0.5,
182 -- It's not entirely clear to me why I have to reproduce all of this.
186 -fwarn-missing-signatures
187 -fwarn-name-shadowing
191 -fwarn-incomplete-record-updates
192 -fwarn-monomorphism-restriction
193 -fwarn-unused-do-bind
198 type: exitcode-stdio-1.0
203 -- Additional test dependencies.
206 -- It's not entirely clear to me why I have to reproduce all of this.
210 -fwarn-missing-signatures
211 -fwarn-name-shadowing
215 -fwarn-incomplete-record-updates
216 -fwarn-monomorphism-restriction
217 -fwarn-unused-do-bind
225 -- These won't work without shelltestrunner installed in your
226 -- $PATH. Maybe there is some way to tell Cabal that.
227 test-suite shelltests
228 type: exitcode-stdio-1.0
230 main-is: ShellTests.hs
239 htsn-common >= 0.0.1,
242 groundhog-postgresql >= 0.5,
243 groundhog-sqlite >= 0.5,
257 source-repository head
259 location: http://michael.orlitzky.com/git/htsn-import.git