]> gitweb.michael.orlitzky.com - dead/htsn-import.git/blob - htsn-import.cabal
Add a minor TODO note.
[dead/htsn-import.git] / htsn-import.cabal
1 name: htsn-import
2 version: 0.0.5
3 cabal-version: >= 1.8
4 author: Michael Orlitzky
5 maintainer: Michael Orlitzky <michael@orlitzky.com>
6 category: Utils
7 license: GPL-3
8 license-file: doc/LICENSE
9 build-type: Simple
10 extra-source-files:
11 doc/dbschema/*.png
12 doc/htsn-importrc.example
13 doc/man1/htsn-import.1
14 doc/README.dbschema
15 doc/README.schemagen
16 doc/TODO
17 makefile
18 schema/*.dtd
19 schemagen/Auto_Racing_Schedule_XML/*.xml
20 schemagen/Heartbeat/*.xml
21 schemagen/injuriesxml/*.xml
22 schemagen/Injuries_Detail_XML/*.xml
23 schemagen/newsxml/*.xml
24 schemagen/Odds_XML/*.xml
25 schemagen/scoresxml/*.xml
26 schemagen/weatherxml/*.xml
27 test/shell/*.test
28 test/xml/*.xml
29 test/xml/*.dtd
30 synopsis:
31 Import XML files from The Sports Network into an RDBMS.
32 description:
33 /Usage/:
34 .
35 @
36 htsn-import [OPTIONS] [FILES]
37 @
38 .
39 The Sports Network <http://www.sportsnetwork.com/> offers an XML feed
40 containing various sports news and statistics. Our sister program
41 /htsn/ is capable of retrieving the feed and saving the individual
42 XML documents contained therein. But what to do with them?
43 .
44 The purpose of /htsn-import/ is to take these XML documents and
45 get them into something we can use, a relational database management
46 system (RDBMS), loosely known as a SQL database. The structure of
47 relational database, is, well, relational, and the feed XML is not. So
48 there is some work to do before the data can be inserted.
49 .
50 First, we must parse the XML. Each supported document type (see below)
51 has a full pickle/unpickle implementation (\"pickle\" is simply a
52 synonym for serialize here). That means that we parse the entire
53 document into a data structure, and if we pickle (serialize) that data
54 structure, we get the exact same XML document tha we started with.
55 .
56 This is important for two reasons. First, it serves as a second level
57 of validation. The first validation is performed by the XML parser,
58 but if that succeeds and unpicking fails, we know that something is
59 fishy. Second, we don't ever want to be surprised by some new element
60 or attribute showing up in the XML. The fact that we can unpickle the
61 whole thing now means that we won't be surprised in the future.
62 .
63 The aforementioned feature is especially important because we
64 automatically migrate the database schema every time we import a
65 document. If you attempt to import a \"newsxml.dtd\" document, all
66 database objects relating to the news will be created if they do not
67 exist. We don't want the schema to change out from under us without
68 warning, so it's important that no XML be parsed that would result in
69 a different schema than we had previously. Since we can
70 pickle/unpickle everything already, this should be impossible.
71 .
72 Examples and usage documentation are available in the man page.
73
74 executable htsn-import
75 build-depends:
76 base >= 4.6 && < 5,
77 cmdargs >= 0.10.6,
78 configurator >= 0.2,
79 directory >= 1.2,
80 filepath >= 1.3,
81 hslogger >= 1.2,
82 htsn-common >= 0.0.1,
83 hxt >= 9.3,
84 groundhog >= 0.5,
85 groundhog-postgresql >= 0.5,
86 groundhog-sqlite >= 0.5,
87 groundhog-th >= 0.5,
88 MissingH >= 1.2,
89 old-locale >= 1,
90 tasty >= 0.8,
91 tasty-hunit >= 0.8,
92 time >= 1.4,
93 transformers >= 0.3,
94 tuple >= 0.2
95
96 main-is:
97 Main.hs
98
99 hs-source-dirs:
100 src/
101
102 other-modules:
103 Backend
104 CommandLine
105 Configuration
106 ConnectionString
107 ExitCodes
108 OptionalConfiguration
109 TSN.Codegen
110 TSN.Database
111 TSN.DbImport
112 TSN.Parse
113 TSN.Picklers
114 TSN.XmlImport
115 TSN.XML.AutoRacingSchedule
116 TSN.XML.GameInfo
117 TSN.XML.Heartbeat
118 TSN.XML.Injuries
119 TSN.XML.InjuriesDetail
120 TSN.XML.News
121 TSN.XML.Odds
122 TSN.XML.Scores
123 TSN.XML.SportInfo
124 TSN.XML.Weather
125 Xml
126
127 ghc-options:
128 -Wall
129 -fwarn-hi-shadowing
130 -fwarn-missing-signatures
131 -fwarn-name-shadowing
132 -fwarn-orphans
133 -fwarn-type-defaults
134 -fwarn-tabs
135 -fwarn-incomplete-record-updates
136 -fwarn-monomorphism-restriction
137 -fwarn-unused-do-bind
138 -O2
139
140 ghc-prof-options:
141 -prof
142 -fprof-auto
143 -fprof-cafs
144 -- The following unbreak profiling with template haskell. We have
145 -- to build the program twice; once without profile and again with
146 -- these flags.
147 -hisuf hi_p
148 -osuf o_p
149
150
151 test-suite testsuite
152 type: exitcode-stdio-1.0
153 hs-source-dirs: src test
154 main-is: TestSuite.hs
155 build-depends:
156 base >= 4.6 && < 5,
157 cmdargs >= 0.10.6,
158 configurator >= 0.2,
159 directory >= 1.2,
160 filepath >= 1.3,
161 hslogger >= 1.2,
162 htsn-common >= 0.0.1,
163 hxt >= 9.3,
164 groundhog >= 0.5,
165 groundhog-postgresql >= 0.5,
166 groundhog-sqlite >= 0.5,
167 groundhog-th >= 0.5,
168 MissingH >= 1.2,
169 old-locale >= 1,
170 tasty >= 0.8,
171 tasty-hunit >= 0.8,
172 time >= 1.4,
173 transformers >= 0.3,
174 tuple >= 0.2
175
176 -- It's not entirely clear to me why I have to reproduce all of this.
177 ghc-options:
178 -Wall
179 -fwarn-hi-shadowing
180 -fwarn-missing-signatures
181 -fwarn-name-shadowing
182 -fwarn-orphans
183 -fwarn-type-defaults
184 -fwarn-tabs
185 -fwarn-incomplete-record-updates
186 -fwarn-monomorphism-restriction
187 -fwarn-unused-do-bind
188 -O2
189
190
191 test-suite doctests
192 type: exitcode-stdio-1.0
193 hs-source-dirs: test
194 main-is: Doctests.hs
195 build-depends:
196 base >= 4.6 && < 5,
197 -- Additional test dependencies.
198 doctest >= 0.9
199
200 -- It's not entirely clear to me why I have to reproduce all of this.
201 ghc-options:
202 -Wall
203 -fwarn-hi-shadowing
204 -fwarn-missing-signatures
205 -fwarn-name-shadowing
206 -fwarn-orphans
207 -fwarn-type-defaults
208 -fwarn-tabs
209 -fwarn-incomplete-record-updates
210 -fwarn-monomorphism-restriction
211 -fwarn-unused-do-bind
212 -rtsopts
213 -threaded
214 -optc-O3
215 -optc-march=native
216 -O2
217
218
219 -- These won't work without shelltestrunner installed in your
220 -- $PATH. Maybe there is some way to tell Cabal that.
221 test-suite shelltests
222 type: exitcode-stdio-1.0
223 hs-source-dirs: test
224 main-is: ShellTests.hs
225
226 build-depends:
227 base >= 4.6 && < 5,
228 cmdargs >= 0.10.6,
229 configurator >= 0.2,
230 directory >= 1.2,
231 filepath >= 1.3,
232 hslogger >= 1.2,
233 htsn-common >= 0.0.1,
234 hxt >= 9.3,
235 groundhog >= 0.5,
236 groundhog-postgresql >= 0.5,
237 groundhog-sqlite >= 0.5,
238 groundhog-th >= 0.5,
239 MissingH >= 1.2,
240 old-locale >= 1,
241 process >= 1.1,
242 tasty >= 0.8,
243 tasty-hunit >= 0.8,
244 time >= 1.4,
245 transformers >= 0.3,
246 tuple >= 0.2
247
248
249
250 source-repository head
251 type: git
252 location: http://michael.orlitzky.com/git/htsn-import.git
253 branch: master