]>
gitweb.michael.orlitzky.com - dead/htsn-import.git/log
Michael Orlitzky [Tue, 14 Jan 2014 00:48:43 +0000 (19:48 -0500)]
Fix remaining compiler warnings.
Michael Orlitzky [Tue, 14 Jan 2014 00:20:52 +0000 (19:20 -0500)]
Fixing compiler warnings by exporting stuff that shouldn't be exported (woo).
Michael Orlitzky [Tue, 14 Jan 2014 00:12:59 +0000 (19:12 -0500)]
Minor makefile fix.
Michael Orlitzky [Tue, 14 Jan 2014 00:12:15 +0000 (19:12 -0500)]
Add the newsxml DB schema PDF.
Michael Orlitzky [Tue, 14 Jan 2014 00:02:59 +0000 (19:02 -0500)]
Add some more documentation to News.hs.
Michael Orlitzky [Mon, 13 Jan 2014 23:59:38 +0000 (18:59 -0500)]
Finish documenting TSN.XML.News and fix all compiler warnings therein.
Michael Orlitzky [Mon, 13 Jan 2014 21:57:38 +0000 (16:57 -0500)]
Add the test file needed by the previous commit.
Michael Orlitzky [Mon, 13 Jan 2014 21:57:01 +0000 (16:57 -0500)]
Add a test for unpickling large Odds_XML files.
Disable DTD validation so the new test passes.
Add a TODO item to re-enable validation.
Michael Orlitzky [Mon, 13 Jan 2014 20:55:13 +0000 (15:55 -0500)]
Add the PDF of the injuries_detail DB schema.
Michael Orlitzky [Mon, 13 Jan 2014 20:39:47 +0000 (15:39 -0500)]
Add more files to 'make clean'.
Michael Orlitzky [Mon, 13 Jan 2014 20:39:03 +0000 (15:39 -0500)]
Remove unused import.
Michael Orlitzky [Mon, 13 Jan 2014 18:50:52 +0000 (13:50 -0500)]
Fix unpickling of non-integer team_ids in TSN.XML.Odds (with tests).
Michael Orlitzky [Mon, 13 Jan 2014 18:41:06 +0000 (13:41 -0500)]
Fix unpickling of texttual over/under lines.
Michael Orlitzky [Mon, 13 Jan 2014 18:40:52 +0000 (13:40 -0500)]
Remove unused imports.
Michael Orlitzky [Mon, 13 Jan 2014 18:40:30 +0000 (13:40 -0500)]
Use (xpOption xpText) instead of xpText0.
Michael Orlitzky [Mon, 13 Jan 2014 17:57:49 +0000 (12:57 -0500)]
Make the text of a news URL optional.
Michael Orlitzky [Mon, 13 Jan 2014 17:41:01 +0000 (12:41 -0500)]
Fix pickle/unpickle of non-interger team_ids and add a test case for it.
Michael Orlitzky [Mon, 13 Jan 2014 15:11:35 +0000 (10:11 -0500)]
Add a sample Injuries_Detail_XML with a noninteger Team_ID.
Michael Orlitzky [Mon, 13 Jan 2014 07:10:12 +0000 (02:10 -0500)]
Add the DB schema PDF for injuries documents.
Michael Orlitzky [Mon, 13 Jan 2014 05:37:02 +0000 (00:37 -0500)]
Update the schemagen README.
Michael Orlitzky [Mon, 13 Jan 2014 05:36:42 +0000 (00:36 -0500)]
Add "correct" versions of all DTDs under schema/.
Michael Orlitzky [Mon, 13 Jan 2014 05:05:35 +0000 (00:05 -0500)]
Remove an unused import.
Michael Orlitzky [Mon, 13 Jan 2014 05:01:16 +0000 (00:01 -0500)]
Add a doctest suite with a big-ass caveat.
Michael Orlitzky [Mon, 13 Jan 2014 04:16:13 +0000 (23:16 -0500)]
Update documentation.
Michael Orlitzky [Mon, 13 Jan 2014 04:01:55 +0000 (23:01 -0500)]
Remove unused XmlPickler instances (this might need to be revisited if regular-xmlpickler sees a new release).
Michael Orlitzky [Mon, 13 Jan 2014 03:54:15 +0000 (22:54 -0500)]
Simplify some tests by passing a pickler instead of relying on a XmlPickler instance.
Michael Orlitzky [Sun, 12 Jan 2014 23:33:00 +0000 (18:33 -0500)]
Add and update documentation.
Michael Orlitzky [Sun, 12 Jan 2014 23:32:21 +0000 (18:32 -0500)]
Add a description to the cabal file, and begin to define extra modules/source files that will be needed for a release.
Michael Orlitzky [Sun, 12 Jan 2014 22:47:59 +0000 (17:47 -0500)]
Add a preliminary man page.
Michael Orlitzky [Sun, 12 Jan 2014 05:32:45 +0000 (00:32 -0500)]
Rename the "injuries" and "injuries_detail" tables to "injuries_listings" and "injuries_detail_player_listings" respectively.
Michael Orlitzky [Sun, 12 Jan 2014 05:01:54 +0000 (00:01 -0500)]
USe lowerCaseSuffixNamingStyle as our base in TSN.Codegen.
Michael Orlitzky [Sun, 12 Jan 2014 05:01:35 +0000 (00:01 -0500)]
Don't remove imported files by default.
Michael Orlitzky [Sat, 11 Jan 2014 23:44:27 +0000 (18:44 -0500)]
Beat TSN.XML.Odds into relational submission; file import actually works now.
Michael Orlitzky [Sat, 11 Jan 2014 23:43:50 +0000 (18:43 -0500)]
Add some more customization to our codegen config.
Michael Orlitzky [Sat, 11 Jan 2014 23:42:59 +0000 (18:42 -0500)]
Use run_dbmigrate instead of runMigration... everywhere.
Michael Orlitzky [Sat, 11 Jan 2014 23:34:26 +0000 (18:34 -0500)]
Add the insert_xml_or_select function to TSN.XmlImport and use it in TSN.XML.News.
Michael Orlitzky [Sat, 11 Jan 2014 23:33:23 +0000 (18:33 -0500)]
Add run_dbmigrate to TSN.DbImport.
Michael Orlitzky [Sat, 11 Jan 2014 08:37:44 +0000 (03:37 -0500)]
Get the Odds module to compile and pass tests under the new regime.
Michael Orlitzky [Sat, 11 Jan 2014 07:07:35 +0000 (02:07 -0500)]
Add missing TSN.XmlImport module.
Michael Orlitzky [Sat, 11 Jan 2014 04:10:24 +0000 (23:10 -0500)]
Rewrite everything to use XmlImport/DbImport classes making things much more easy to understand. The docs need an update now.
Michael Orlitzky [Fri, 10 Jan 2014 23:29:19 +0000 (18:29 -0500)]
Begin reworking TSN.XML.Odds for the new inferred DTDs.
Michael Orlitzky [Fri, 10 Jan 2014 21:53:12 +0000 (16:53 -0500)]
Simplify parse_opts now that we can validate against the DTDs.
Michael Orlitzky [Fri, 10 Jan 2014 21:48:50 +0000 (16:48 -0500)]
Add the hand-crafted Odds_XML.dtd.
Michael Orlitzky [Fri, 10 Jan 2014 21:48:13 +0000 (16:48 -0500)]
Add a note to the README about the hand-crafted schemas.
Michael Orlitzky [Fri, 10 Jan 2014 21:47:31 +0000 (16:47 -0500)]
Replace the auto-generated Odds_XML.dtd used in tests with the hand-crafted one.
Michael Orlitzky [Fri, 10 Jan 2014 21:46:56 +0000 (16:46 -0500)]
Change the Odds database type to 'Odds' in Main.hs.
Michael Orlitzky [Fri, 10 Jan 2014 04:59:38 +0000 (23:59 -0500)]
Begin validating the XML.
Fix a bug in the Injuries pickling.
Michael Orlitzky [Fri, 10 Jan 2014 04:53:27 +0000 (23:53 -0500)]
Update the test TSN DTDs with the inferred ones.
Michael Orlitzky [Fri, 10 Jan 2014 04:52:35 +0000 (23:52 -0500)]
Add an embedded type to TSN.XML.Injuries to allow it to parse the new inferred DTD.
Michael Orlitzky [Fri, 10 Jan 2014 04:52:10 +0000 (23:52 -0500)]
Remove unused BangPatterns pragmas.
Michael Orlitzky [Fri, 10 Jan 2014 04:17:02 +0000 (23:17 -0500)]
Add an empty man page.
Michael Orlitzky [Fri, 10 Jan 2014 04:15:45 +0000 (23:15 -0500)]
Make NewsTeams and NewsLocations unique.
Create join tables between "news", "news_locations", and "news_teams".
Rename the non-XML version of News.Message to "News".
Michael Orlitzky [Fri, 10 Jan 2014 04:04:15 +0000 (23:04 -0500)]
Change the order of init_logging's arguments.
Michael Orlitzky [Thu, 9 Jan 2014 00:09:01 +0000 (19:09 -0500)]
Change the XML-Schema-learner executable name.
Michael Orlitzky [Wed, 8 Jan 2014 01:34:49 +0000 (20:34 -0500)]
Add a new example of newsxml.
Michael Orlitzky [Tue, 7 Jan 2014 02:28:59 +0000 (21:28 -0500)]
Add more newsxml samples required for an accurate DTD.
Michael Orlitzky [Tue, 7 Jan 2014 02:24:31 +0000 (21:24 -0500)]
Add two new injuriesxml files needed to generate a proper schema.
Michael Orlitzky [Mon, 6 Jan 2014 18:49:05 +0000 (13:49 -0500)]
Begin work on the automatic schema generation.
Michael Orlitzky [Thu, 2 Jan 2014 17:00:42 +0000 (12:00 -0500)]
Fix hlint suggestions.
Michael Orlitzky [Thu, 2 Jan 2014 15:59:51 +0000 (10:59 -0500)]
Update the newsxml DTD to make the Editor optional.
Add a second test newsxml document which has an Editor element.
Replicate News tests for the Editor-having document.
Michael Orlitzky [Thu, 2 Jan 2014 05:13:39 +0000 (00:13 -0500)]
Add tests for Odds pickle/unpickle.
Michael Orlitzky [Thu, 2 Jan 2014 05:12:41 +0000 (00:12 -0500)]
Use 'def' in a few places where default values are required.
Michael Orlitzky [Thu, 2 Jan 2014 05:12:06 +0000 (00:12 -0500)]
Add separate 'unpickleable' tests to the existing XML modules.
Michael Orlitzky [Thu, 2 Jan 2014 05:10:54 +0000 (00:10 -0500)]
Add 'unpickleable' to Xml for use in tests.
Michael Orlitzky [Thu, 2 Jan 2014 01:08:28 +0000 (20:08 -0500)]
Add TSN.XML.Odds which can (only, for the moment) parse Odds_XML.xml.
Michael Orlitzky [Thu, 2 Jan 2014 01:02:24 +0000 (20:02 -0500)]
Remove the XmlPickler instance for Bool.
Use xpPrim instead of the (removed) xpickle for Bools.
Replace xpPrim with xpInt where possible.
Michael Orlitzky [Thu, 2 Jan 2014 01:00:25 +0000 (20:00 -0500)]
Don't call clean before every compile (they're getting slow).
Michael Orlitzky [Wed, 1 Jan 2014 23:10:04 +0000 (18:10 -0500)]
Rename doctypeA to dtdnameA.
Michael Orlitzky [Wed, 1 Jan 2014 19:19:35 +0000 (14:19 -0500)]
Rename the ImportResult constructors.
Add some code documentation.
Wrap the DTD name (String) in a newtype, DtdName.
Michael Orlitzky [Wed, 1 Jan 2014 05:15:27 +0000 (00:15 -0500)]
Create an ImportResult type and refactor things around it.
Add Heartbeat support.
Michael Orlitzky [Wed, 1 Jan 2014 02:07:31 +0000 (21:07 -0500)]
Add code to main to delete successfully imported files.
Michael Orlitzky [Tue, 31 Dec 2013 22:11:35 +0000 (17:11 -0500)]
Add a test for the correct field names in TSN.Xml.News.
Michael Orlitzky [Tue, 31 Dec 2013 21:40:47 +0000 (16:40 -0500)]
Move the XML modules into the XML subdirectory.
Don't save the default <message> fields along with the news.
Make the makefile a little more accurate.
Michael Orlitzky [Tue, 31 Dec 2013 21:31:17 +0000 (16:31 -0500)]
Get the news import working more or less how it's supposed to.
Michael Orlitzky [Tue, 31 Dec 2013 21:30:59 +0000 (16:30 -0500)]
Add the TSN.Codegen module to help Groundhog out.
Michael Orlitzky [Tue, 31 Dec 2013 19:41:03 +0000 (14:41 -0500)]
Replace the FromXml class with ToFromXml and add a method to it.
Michael Orlitzky [Tue, 31 Dec 2013 19:36:20 +0000 (14:36 -0500)]
Add Heartbeat XML and DTD.
Michael Orlitzky [Tue, 31 Dec 2013 17:14:25 +0000 (12:14 -0500)]
Add an Odds_XML.xml example.
Michael Orlitzky [Tue, 31 Dec 2013 16:37:23 +0000 (11:37 -0500)]
Add the FromXml type family.
Michael Orlitzky [Tue, 31 Dec 2013 14:19:23 +0000 (09:19 -0500)]
Add a DbImport class implementing the import interface.
Write import_injuries, import_injuries_detail, and import_news for the new type class.
Move DbImport instances into the module where the type is defined.
Get the News import working (temporarily) via unsafeCoerce.
Michael Orlitzky [Mon, 30 Dec 2013 20:03:37 +0000 (15:03 -0500)]
Reorganize the import_foo functions so that the database choice comes sooner and everything afterwards runs in a Groundhog monad.
Michael Orlitzky [Mon, 30 Dec 2013 17:42:03 +0000 (12:42 -0500)]
Get pickling (but not insertion) working for TSN.News.
Michael Orlitzky [Mon, 30 Dec 2013 15:09:48 +0000 (10:09 -0500)]
Add a TSN.News module and get it to compile.
Michael Orlitzky [Mon, 30 Dec 2013 14:15:39 +0000 (09:15 -0500)]
Add a weatherxml file from the feed.
Michael Orlitzky [Mon, 30 Dec 2013 13:42:10 +0000 (08:42 -0500)]
Add two more DTDs for tables we'll create in the future.
Michael Orlitzky [Mon, 30 Dec 2013 13:34:21 +0000 (08:34 -0500)]
Add DTDs for the three existing XML files.
Michael Orlitzky [Mon, 30 Dec 2013 03:55:50 +0000 (22:55 -0500)]
Add a tasty test suite and two tests for the existing XML modules.
Michael Orlitzky [Mon, 30 Dec 2013 03:55:06 +0000 (22:55 -0500)]
Add a special pickler for the wacky team_ids.
Michael Orlitzky [Sun, 29 Dec 2013 23:55:30 +0000 (18:55 -0500)]
Provide a default connection string (for sqlite).
Implement the backend choice; you can now really choose your database.
Michael Orlitzky [Sun, 29 Dec 2013 07:16:52 +0000 (02:16 -0500)]
Make it really import the XML files from the command line.
Jump through a bunch of hoops to get proper status messages (and error reporting) from the arrow composition.
Michael Orlitzky [Sun, 29 Dec 2013 01:19:39 +0000 (20:19 -0500)]
Add a backend configuration option.
Add command-line parsing.
Depend on the new library htsn-common.
Remove the TODO since it was empty.
Michael Orlitzky [Sat, 28 Dec 2013 17:36:19 +0000 (12:36 -0500)]
Add a TODO.
Michael Orlitzky [Sat, 28 Dec 2013 17:35:34 +0000 (12:35 -0500)]
Add copies of htsn's Logging and Terminal modules (to be replaced later).
Michael Orlitzky [Sat, 28 Dec 2013 17:35:14 +0000 (12:35 -0500)]
Bump the cabal file with new deps.
Michael Orlitzky [Sat, 28 Dec 2013 17:35:01 +0000 (12:35 -0500)]
Add (Optional)Configuration modules.
Michael Orlitzky [Sat, 28 Dec 2013 17:12:02 +0000 (12:12 -0500)]
Rename to htsn-import for simplicity.
Michael Orlitzky [Sat, 28 Dec 2013 03:14:29 +0000 (22:14 -0500)]
Implement database insertion (generically!).
Michael Orlitzky [Sat, 28 Dec 2013 03:14:05 +0000 (22:14 -0500)]
Uncomment DTDs for test XML.
Michael Orlitzky [Sat, 28 Dec 2013 03:13:22 +0000 (22:13 -0500)]
Remove pretty-show and transformers dependencies.
Michael Orlitzky [Fri, 27 Dec 2013 14:03:54 +0000 (09:03 -0500)]
Comment out the DTD in Injuries_Detail_XML.xml.
Switch from persistent to groundhog.
Remove Uncurry.hs in favor of Data.Tuple.Curry.
Add TSN.InjuriesDetail parsing.
Add a TSN.Picklers module to (un)pickle various stuffs.