]> gitweb.michael.orlitzky.com - dead/htsn-import.git/log
dead/htsn-import.git
10 years agoFinish documenting TSN.XML.News and fix all compiler warnings therein.
Michael Orlitzky [Mon, 13 Jan 2014 23:59:38 +0000 (18:59 -0500)]
Finish documenting TSN.XML.News and fix all compiler warnings therein.

10 years agoAdd the test file needed by the previous commit.
Michael Orlitzky [Mon, 13 Jan 2014 21:57:38 +0000 (16:57 -0500)]
Add the test file needed by the previous commit.

10 years agoAdd a test for unpickling large Odds_XML files.
Michael Orlitzky [Mon, 13 Jan 2014 21:57:01 +0000 (16:57 -0500)]
Add a test for unpickling large Odds_XML files.
Disable DTD validation so the new test passes.
Add a TODO item to re-enable validation.

10 years agoAdd the PDF of the injuries_detail DB schema.
Michael Orlitzky [Mon, 13 Jan 2014 20:55:13 +0000 (15:55 -0500)]
Add the PDF of the injuries_detail DB schema.

10 years agoAdd more files to 'make clean'.
Michael Orlitzky [Mon, 13 Jan 2014 20:39:47 +0000 (15:39 -0500)]
Add more files to 'make clean'.

10 years agoRemove unused import.
Michael Orlitzky [Mon, 13 Jan 2014 20:39:03 +0000 (15:39 -0500)]
Remove unused import.

10 years agoFix unpickling of non-integer team_ids in TSN.XML.Odds (with tests).
Michael Orlitzky [Mon, 13 Jan 2014 18:50:52 +0000 (13:50 -0500)]
Fix unpickling of non-integer team_ids in TSN.XML.Odds (with tests).

10 years agoFix unpickling of texttual over/under lines.
Michael Orlitzky [Mon, 13 Jan 2014 18:41:06 +0000 (13:41 -0500)]
Fix unpickling of texttual over/under lines.

10 years agoRemove unused imports.
Michael Orlitzky [Mon, 13 Jan 2014 18:40:52 +0000 (13:40 -0500)]
Remove unused imports.

10 years agoUse (xpOption xpText) instead of xpText0.
Michael Orlitzky [Mon, 13 Jan 2014 18:40:30 +0000 (13:40 -0500)]
Use (xpOption xpText) instead of xpText0.

10 years agoMake the text of a news URL optional.
Michael Orlitzky [Mon, 13 Jan 2014 17:57:49 +0000 (12:57 -0500)]
Make the text of a news URL optional.

10 years agoFix pickle/unpickle of non-interger team_ids and add a test case for it.
Michael Orlitzky [Mon, 13 Jan 2014 17:41:01 +0000 (12:41 -0500)]
Fix pickle/unpickle of non-interger team_ids and add a test case for it.

10 years agoAdd a sample Injuries_Detail_XML with a noninteger Team_ID.
Michael Orlitzky [Mon, 13 Jan 2014 15:11:35 +0000 (10:11 -0500)]
Add a sample Injuries_Detail_XML with a noninteger Team_ID.

10 years agoAdd the DB schema PDF for injuries documents.
Michael Orlitzky [Mon, 13 Jan 2014 07:10:12 +0000 (02:10 -0500)]
Add the DB schema PDF for injuries documents.

10 years agoUpdate the schemagen README.
Michael Orlitzky [Mon, 13 Jan 2014 05:37:02 +0000 (00:37 -0500)]
Update the schemagen README.

10 years agoAdd "correct" versions of all DTDs under schema/.
Michael Orlitzky [Mon, 13 Jan 2014 05:36:42 +0000 (00:36 -0500)]
Add "correct" versions of all DTDs under schema/.

10 years agoRemove an unused import.
Michael Orlitzky [Mon, 13 Jan 2014 05:05:35 +0000 (00:05 -0500)]
Remove an unused import.

10 years agoAdd a doctest suite with a big-ass caveat.
Michael Orlitzky [Mon, 13 Jan 2014 05:01:16 +0000 (00:01 -0500)]
Add a doctest suite with a big-ass caveat.

10 years agoUpdate documentation.
Michael Orlitzky [Mon, 13 Jan 2014 04:16:13 +0000 (23:16 -0500)]
Update documentation.

10 years agoRemove unused XmlPickler instances (this might need to be revisited if regular-xmlpic...
Michael Orlitzky [Mon, 13 Jan 2014 04:01:55 +0000 (23:01 -0500)]
Remove unused XmlPickler instances (this might need to be revisited if regular-xmlpickler sees a new release).

10 years agoSimplify some tests by passing a pickler instead of relying on a XmlPickler instance.
Michael Orlitzky [Mon, 13 Jan 2014 03:54:15 +0000 (22:54 -0500)]
Simplify some tests by passing a pickler instead of relying on a XmlPickler instance.

10 years agoAdd and update documentation.
Michael Orlitzky [Sun, 12 Jan 2014 23:33:00 +0000 (18:33 -0500)]
Add and update documentation.

10 years agoAdd a description to the cabal file, and begin to define extra modules/source files...
Michael Orlitzky [Sun, 12 Jan 2014 23:32:21 +0000 (18:32 -0500)]
Add a description to the cabal file, and begin to define extra modules/source files that will be needed for a release.

10 years agoAdd a preliminary man page.
Michael Orlitzky [Sun, 12 Jan 2014 22:47:59 +0000 (17:47 -0500)]
Add a preliminary man page.

10 years agoRename the "injuries" and "injuries_detail" tables to "injuries_listings" and "injuri...
Michael Orlitzky [Sun, 12 Jan 2014 05:32:45 +0000 (00:32 -0500)]
Rename the "injuries" and "injuries_detail" tables to "injuries_listings" and "injuries_detail_player_listings" respectively.

10 years agoUSe lowerCaseSuffixNamingStyle as our base in TSN.Codegen.
Michael Orlitzky [Sun, 12 Jan 2014 05:01:54 +0000 (00:01 -0500)]
USe lowerCaseSuffixNamingStyle as our base in TSN.Codegen.

10 years agoDon't remove imported files by default.
Michael Orlitzky [Sun, 12 Jan 2014 05:01:35 +0000 (00:01 -0500)]
Don't remove imported files by default.

10 years agoBeat TSN.XML.Odds into relational submission; file import actually works now.
Michael Orlitzky [Sat, 11 Jan 2014 23:44:27 +0000 (18:44 -0500)]
Beat TSN.XML.Odds into relational submission; file import actually works now.

10 years agoAdd some more customization to our codegen config.
Michael Orlitzky [Sat, 11 Jan 2014 23:43:50 +0000 (18:43 -0500)]
Add some more customization to our codegen config.

10 years agoUse run_dbmigrate instead of runMigration... everywhere.
Michael Orlitzky [Sat, 11 Jan 2014 23:42:59 +0000 (18:42 -0500)]
Use run_dbmigrate instead of runMigration... everywhere.

10 years agoAdd the insert_xml_or_select function to TSN.XmlImport and use it in TSN.XML.News.
Michael Orlitzky [Sat, 11 Jan 2014 23:34:26 +0000 (18:34 -0500)]
Add the insert_xml_or_select function to TSN.XmlImport and use it in TSN.XML.News.

10 years agoAdd run_dbmigrate to TSN.DbImport.
Michael Orlitzky [Sat, 11 Jan 2014 23:33:23 +0000 (18:33 -0500)]
Add run_dbmigrate to TSN.DbImport.

10 years agoGet the Odds module to compile and pass tests under the new regime.
Michael Orlitzky [Sat, 11 Jan 2014 08:37:44 +0000 (03:37 -0500)]
Get the Odds module to compile and pass tests under the new regime.

10 years agoAdd missing TSN.XmlImport module.
Michael Orlitzky [Sat, 11 Jan 2014 07:07:35 +0000 (02:07 -0500)]
Add missing TSN.XmlImport module.

10 years agoRewrite everything to use XmlImport/DbImport classes making things much more easy...
Michael Orlitzky [Sat, 11 Jan 2014 04:10:24 +0000 (23:10 -0500)]
Rewrite everything to use XmlImport/DbImport classes making things much more easy to understand. The docs need an update now.

10 years agoBegin reworking TSN.XML.Odds for the new inferred DTDs.
Michael Orlitzky [Fri, 10 Jan 2014 23:29:19 +0000 (18:29 -0500)]
Begin reworking TSN.XML.Odds for the new inferred DTDs.

10 years agoSimplify parse_opts now that we can validate against the DTDs.
Michael Orlitzky [Fri, 10 Jan 2014 21:53:12 +0000 (16:53 -0500)]
Simplify parse_opts now that we can validate against the DTDs.

10 years agoAdd the hand-crafted Odds_XML.dtd.
Michael Orlitzky [Fri, 10 Jan 2014 21:48:50 +0000 (16:48 -0500)]
Add the hand-crafted Odds_XML.dtd.

10 years agoAdd a note to the README about the hand-crafted schemas.
Michael Orlitzky [Fri, 10 Jan 2014 21:48:13 +0000 (16:48 -0500)]
Add a note to the README about the hand-crafted schemas.

10 years agoReplace the auto-generated Odds_XML.dtd used in tests with the hand-crafted one.
Michael Orlitzky [Fri, 10 Jan 2014 21:47:31 +0000 (16:47 -0500)]
Replace the auto-generated Odds_XML.dtd used in tests with the hand-crafted one.

10 years agoChange the Odds database type to 'Odds' in Main.hs.
Michael Orlitzky [Fri, 10 Jan 2014 21:46:56 +0000 (16:46 -0500)]
Change the Odds database type to 'Odds' in Main.hs.

10 years agoBegin validating the XML.
Michael Orlitzky [Fri, 10 Jan 2014 04:59:38 +0000 (23:59 -0500)]
Begin validating the XML.
Fix a bug in the Injuries pickling.

10 years agoUpdate the test TSN DTDs with the inferred ones.
Michael Orlitzky [Fri, 10 Jan 2014 04:53:27 +0000 (23:53 -0500)]
Update the test TSN DTDs with the inferred ones.

10 years agoAdd an embedded type to TSN.XML.Injuries to allow it to parse the new inferred DTD.
Michael Orlitzky [Fri, 10 Jan 2014 04:52:35 +0000 (23:52 -0500)]
Add an embedded type to TSN.XML.Injuries to allow it to parse the new inferred DTD.

10 years agoRemove unused BangPatterns pragmas.
Michael Orlitzky [Fri, 10 Jan 2014 04:52:10 +0000 (23:52 -0500)]
Remove unused BangPatterns pragmas.

10 years agoAdd an empty man page.
Michael Orlitzky [Fri, 10 Jan 2014 04:17:02 +0000 (23:17 -0500)]
Add an empty man page.

10 years agoMake NewsTeams and NewsLocations unique.
Michael Orlitzky [Fri, 10 Jan 2014 04:15:45 +0000 (23:15 -0500)]
Make NewsTeams and NewsLocations unique.
Create join tables between "news", "news_locations", and "news_teams".
Rename the non-XML version of News.Message to "News".

10 years agoChange the order of init_logging's arguments.
Michael Orlitzky [Fri, 10 Jan 2014 04:04:15 +0000 (23:04 -0500)]
Change the order of init_logging's arguments.

10 years agoChange the XML-Schema-learner executable name.
Michael Orlitzky [Thu, 9 Jan 2014 00:09:01 +0000 (19:09 -0500)]
Change the XML-Schema-learner executable name.

10 years agoAdd a new example of newsxml.
Michael Orlitzky [Wed, 8 Jan 2014 01:34:49 +0000 (20:34 -0500)]
Add a new example of newsxml.

10 years agoAdd more newsxml samples required for an accurate DTD.
Michael Orlitzky [Tue, 7 Jan 2014 02:28:59 +0000 (21:28 -0500)]
Add more newsxml samples required for an accurate DTD.

10 years agoAdd two new injuriesxml files needed to generate a proper schema.
Michael Orlitzky [Tue, 7 Jan 2014 02:24:31 +0000 (21:24 -0500)]
Add two new injuriesxml files needed to generate a proper schema.

10 years agoBegin work on the automatic schema generation.
Michael Orlitzky [Mon, 6 Jan 2014 18:49:05 +0000 (13:49 -0500)]
Begin work on the automatic schema generation.

10 years agoFix hlint suggestions.
Michael Orlitzky [Thu, 2 Jan 2014 17:00:42 +0000 (12:00 -0500)]
Fix hlint suggestions.

10 years agoUpdate the newsxml DTD to make the Editor optional.
Michael Orlitzky [Thu, 2 Jan 2014 15:59:51 +0000 (10:59 -0500)]
Update the newsxml DTD to make the Editor optional.
Add a second test newsxml document which has an Editor element.
Replicate News tests for the Editor-having document.

10 years agoAdd tests for Odds pickle/unpickle.
Michael Orlitzky [Thu, 2 Jan 2014 05:13:39 +0000 (00:13 -0500)]
Add tests for Odds pickle/unpickle.

10 years agoUse 'def' in a few places where default values are required.
Michael Orlitzky [Thu, 2 Jan 2014 05:12:41 +0000 (00:12 -0500)]
Use 'def' in a few places where default values are required.

10 years agoAdd separate 'unpickleable' tests to the existing XML modules.
Michael Orlitzky [Thu, 2 Jan 2014 05:12:06 +0000 (00:12 -0500)]
Add separate 'unpickleable' tests to the existing XML modules.

10 years agoAdd 'unpickleable' to Xml for use in tests.
Michael Orlitzky [Thu, 2 Jan 2014 05:10:54 +0000 (00:10 -0500)]
Add 'unpickleable' to Xml for use in tests.

10 years agoAdd TSN.XML.Odds which can (only, for the moment) parse Odds_XML.xml.
Michael Orlitzky [Thu, 2 Jan 2014 01:08:28 +0000 (20:08 -0500)]
Add TSN.XML.Odds which can (only, for the moment) parse Odds_XML.xml.

10 years agoRemove the XmlPickler instance for Bool.
Michael Orlitzky [Thu, 2 Jan 2014 01:02:24 +0000 (20:02 -0500)]
Remove the XmlPickler instance for Bool.
Use xpPrim instead of the (removed) xpickle for Bools.
Replace xpPrim with xpInt where possible.

10 years agoDon't call clean before every compile (they're getting slow).
Michael Orlitzky [Thu, 2 Jan 2014 01:00:25 +0000 (20:00 -0500)]
Don't call clean before every compile (they're getting slow).

10 years agoRename doctypeA to dtdnameA.
Michael Orlitzky [Wed, 1 Jan 2014 23:10:04 +0000 (18:10 -0500)]
Rename doctypeA to dtdnameA.

10 years agoRename the ImportResult constructors.
Michael Orlitzky [Wed, 1 Jan 2014 19:19:35 +0000 (14:19 -0500)]
Rename the ImportResult constructors.
Add some code documentation.
Wrap the DTD name (String) in a newtype, DtdName.

10 years agoCreate an ImportResult type and refactor things around it.
Michael Orlitzky [Wed, 1 Jan 2014 05:15:27 +0000 (00:15 -0500)]
Create an ImportResult type and refactor things around it.
Add Heartbeat support.

10 years agoAdd code to main to delete successfully imported files.
Michael Orlitzky [Wed, 1 Jan 2014 02:07:31 +0000 (21:07 -0500)]
Add code to main to delete successfully imported files.

10 years agoAdd a test for the correct field names in TSN.Xml.News.
Michael Orlitzky [Tue, 31 Dec 2013 22:11:35 +0000 (17:11 -0500)]
Add a test for the correct field names in TSN.Xml.News.

10 years agoMove the XML modules into the XML subdirectory.
Michael Orlitzky [Tue, 31 Dec 2013 21:40:47 +0000 (16:40 -0500)]
Move the XML modules into the XML subdirectory.
Don't save the default <message> fields along with the news.
Make the makefile a little more accurate.

10 years agoGet the news import working more or less how it's supposed to.
Michael Orlitzky [Tue, 31 Dec 2013 21:31:17 +0000 (16:31 -0500)]
Get the news import working more or less how it's supposed to.

10 years agoAdd the TSN.Codegen module to help Groundhog out.
Michael Orlitzky [Tue, 31 Dec 2013 21:30:59 +0000 (16:30 -0500)]
Add the TSN.Codegen module to help Groundhog out.

10 years agoReplace the FromXml class with ToFromXml and add a method to it.
Michael Orlitzky [Tue, 31 Dec 2013 19:41:03 +0000 (14:41 -0500)]
Replace the FromXml class with ToFromXml and add a method to it.

10 years agoAdd Heartbeat XML and DTD.
Michael Orlitzky [Tue, 31 Dec 2013 19:36:20 +0000 (14:36 -0500)]
Add Heartbeat XML and DTD.

10 years agoAdd an Odds_XML.xml example.
Michael Orlitzky [Tue, 31 Dec 2013 17:14:25 +0000 (12:14 -0500)]
Add an Odds_XML.xml example.

10 years agoAdd the FromXml type family.
Michael Orlitzky [Tue, 31 Dec 2013 16:37:23 +0000 (11:37 -0500)]
Add the FromXml type family.

10 years agoAdd a DbImport class implementing the import interface.
Michael Orlitzky [Tue, 31 Dec 2013 14:19:23 +0000 (09:19 -0500)]
Add a DbImport class implementing the import interface.
Write import_injuries, import_injuries_detail, and import_news for the new type class.
Move DbImport instances into the module where the type is defined.
Get the News import working (temporarily) via unsafeCoerce.

10 years agoReorganize the import_foo functions so that the database choice comes sooner and...
Michael Orlitzky [Mon, 30 Dec 2013 20:03:37 +0000 (15:03 -0500)]
Reorganize the import_foo functions so that the database choice comes sooner and everything afterwards runs in a Groundhog monad.

10 years agoGet pickling (but not insertion) working for TSN.News.
Michael Orlitzky [Mon, 30 Dec 2013 17:42:03 +0000 (12:42 -0500)]
Get pickling (but not insertion) working for TSN.News.

10 years agoAdd a TSN.News module and get it to compile.
Michael Orlitzky [Mon, 30 Dec 2013 15:09:48 +0000 (10:09 -0500)]
Add a TSN.News module and get it to compile.

10 years agoAdd a weatherxml file from the feed.
Michael Orlitzky [Mon, 30 Dec 2013 14:15:39 +0000 (09:15 -0500)]
Add a weatherxml file from the feed.

10 years agoAdd two more DTDs for tables we'll create in the future.
Michael Orlitzky [Mon, 30 Dec 2013 13:42:10 +0000 (08:42 -0500)]
Add two more DTDs for tables we'll create in the future.

10 years agoAdd DTDs for the three existing XML files.
Michael Orlitzky [Mon, 30 Dec 2013 13:34:21 +0000 (08:34 -0500)]
Add DTDs for the three existing XML files.

10 years agoAdd a tasty test suite and two tests for the existing XML modules.
Michael Orlitzky [Mon, 30 Dec 2013 03:55:50 +0000 (22:55 -0500)]
Add a tasty test suite and two tests for the existing XML modules.

10 years agoAdd a special pickler for the wacky team_ids.
Michael Orlitzky [Mon, 30 Dec 2013 03:55:06 +0000 (22:55 -0500)]
Add a special pickler for the wacky team_ids.

10 years agoProvide a default connection string (for sqlite).
Michael Orlitzky [Sun, 29 Dec 2013 23:55:30 +0000 (18:55 -0500)]
Provide a default connection string (for sqlite).
Implement the backend choice; you can now really choose your database.

10 years agoMake it really import the XML files from the command line.
Michael Orlitzky [Sun, 29 Dec 2013 07:16:52 +0000 (02:16 -0500)]
Make it really import the XML files from the command line.
Jump through a bunch of hoops to get proper status messages (and error reporting) from the arrow composition.

10 years agoAdd a backend configuration option.
Michael Orlitzky [Sun, 29 Dec 2013 01:19:39 +0000 (20:19 -0500)]
Add a backend configuration option.
Add command-line parsing.
Depend on the new library htsn-common.
Remove the TODO since it was empty.

10 years agoAdd a TODO.
Michael Orlitzky [Sat, 28 Dec 2013 17:36:19 +0000 (12:36 -0500)]
Add a TODO.

10 years agoAdd copies of htsn's Logging and Terminal modules (to be replaced later).
Michael Orlitzky [Sat, 28 Dec 2013 17:35:34 +0000 (12:35 -0500)]
Add copies of htsn's Logging and Terminal modules (to be replaced later).

10 years agoBump the cabal file with new deps.
Michael Orlitzky [Sat, 28 Dec 2013 17:35:14 +0000 (12:35 -0500)]
Bump the cabal file with new deps.

10 years agoAdd (Optional)Configuration modules.
Michael Orlitzky [Sat, 28 Dec 2013 17:35:01 +0000 (12:35 -0500)]
Add (Optional)Configuration modules.

10 years agoRename to htsn-import for simplicity.
Michael Orlitzky [Sat, 28 Dec 2013 17:12:02 +0000 (12:12 -0500)]
Rename to htsn-import for simplicity.

10 years agoImplement database insertion (generically!).
Michael Orlitzky [Sat, 28 Dec 2013 03:14:29 +0000 (22:14 -0500)]
Implement database insertion (generically!).

10 years agoUncomment DTDs for test XML.
Michael Orlitzky [Sat, 28 Dec 2013 03:14:05 +0000 (22:14 -0500)]
Uncomment DTDs for test XML.

10 years agoRemove pretty-show and transformers dependencies.
Michael Orlitzky [Sat, 28 Dec 2013 03:13:22 +0000 (22:13 -0500)]
Remove pretty-show and transformers dependencies.

10 years agoComment out the DTD in Injuries_Detail_XML.xml.
Michael Orlitzky [Fri, 27 Dec 2013 14:03:54 +0000 (09:03 -0500)]
Comment out the DTD in Injuries_Detail_XML.xml.
Switch from persistent to groundhog.
Remove Uncurry.hs in favor of Data.Tuple.Curry.
Add TSN.InjuriesDetail parsing.
Add a TSN.Picklers module to (un)pickle various stuffs.

10 years agoCommit persistent stuff.
Michael Orlitzky [Thu, 26 Dec 2013 20:25:03 +0000 (15:25 -0500)]
Commit persistent stuff.

10 years agoAdd two more test XML files.
Michael Orlitzky [Thu, 26 Dec 2013 20:24:34 +0000 (15:24 -0500)]
Add two more test XML files.

10 years agoAdd Uncurry.hs.
Michael Orlitzky [Thu, 26 Dec 2013 20:23:36 +0000 (15:23 -0500)]
Add Uncurry.hs.

10 years agoInitial commit, proof-of-concept with Persistent and HXT.
Michael Orlitzky [Wed, 25 Dec 2013 23:10:19 +0000 (18:10 -0500)]
Initial commit, proof-of-concept with Persistent and HXT.