]> gitweb.michael.orlitzky.com - dead/census-tools.git/log
dead/census-tools.git
15 years agoModified the default_option_parser function to return a parser using a VerbatimHelpFo...
Michael Orlitzky [Mon, 16 Nov 2009 01:38:05 +0000 (20:38 -0500)]
Modified the default_option_parser function to return a parser using a VerbatimHelpFormatter.
Updated the scripts using default_option_parser to account for the whitespace changes.

15 years agoAdded an LEHD module, with classes to parse Origin-Destination matrix files.
Michael Orlitzky [Mon, 16 Nov 2009 01:26:15 +0000 (20:26 -0500)]
Added an LEHD module, with classes to parse Origin-Destination matrix files.
Added tests and fixtures for the LEHD module.
Updated the run_tests script to include the new LEHD tests.

15 years agoMoved RecordError in to its own file, Errors.py.
Michael Orlitzky [Mon, 16 Nov 2009 01:01:23 +0000 (20:01 -0500)]
Moved RecordError in to its own file, Errors.py.

15 years agoAdded the LEHD (OnTheMap) technical documentation.
Michael Orlitzky [Mon, 16 Nov 2009 00:09:53 +0000 (19:09 -0500)]
Added the LEHD (OnTheMap) technical documentation.

15 years agoModified the makefile to use the find_file_paths script rather than guessing at some...
Michael Orlitzky [Fri, 13 Nov 2009 02:47:47 +0000 (21:47 -0500)]
Modified the makefile to use the find_file_paths script rather than guessing at some important file locations.

15 years agoRemoved an unnecessary import.
Michael Orlitzky [Fri, 13 Nov 2009 02:23:00 +0000 (21:23 -0500)]
Removed an unnecessary import.

15 years agoMoved the multiple-filename logic inside the find_file_paths function.
Michael Orlitzky [Fri, 13 Nov 2009 02:21:15 +0000 (21:21 -0500)]
Moved the multiple-filename logic inside the find_file_paths function.
Added tests for the FileUtils module, and in particular for the find_file_paths function.
Added two fixtures for the new tests.
Updated the test suite.

15 years agoAdded the find_file_paths script which utilizes the new find_file_paths function...
Michael Orlitzky [Fri, 13 Nov 2009 01:07:42 +0000 (20:07 -0500)]
Added the find_file_paths script which utilizes the new find_file_paths function within the FileUtils module.

15 years agoAdded the find_file_paths function to the FileUtils module.
Michael Orlitzky [Fri, 13 Nov 2009 01:07:11 +0000 (20:07 -0500)]
Added the find_file_paths function to the FileUtils module.

15 years agoAdded the VerbatimHelpFormatter class to the CLI module.
Michael Orlitzky [Fri, 13 Nov 2009 00:59:08 +0000 (19:59 -0500)]
Added the VerbatimHelpFormatter class to the CLI module.

15 years agoFixed a method name in the download_lines function.
Michael Orlitzky [Thu, 12 Nov 2009 23:26:58 +0000 (18:26 -0500)]
Fixed a method name in the download_lines function.

15 years agoModified the makefile to import all of the Summary File 1 data.
Michael Orlitzky [Wed, 11 Nov 2009 21:59:52 +0000 (16:59 -0500)]
Modified the makefile to import all of the Summary File 1 data.

15 years agoModified the Data module and download script to download the Summary File 1 data.
Michael Orlitzky [Wed, 11 Nov 2009 15:57:20 +0000 (10:57 -0500)]
Modified the Data module and download script to download the Summary File 1 data.

15 years agoAdded the path2pop script.
Michael Orlitzky [Wed, 11 Nov 2009 04:40:01 +0000 (23:40 -0500)]
Added the path2pop script.

15 years agoRemoved an unnecessary include.
Michael Orlitzky [Wed, 11 Nov 2009 04:39:44 +0000 (23:39 -0500)]
Removed an unnecessary include.

15 years agoRemoved rows length checks in two places, and replaced them with first-column-not...
Michael Orlitzky [Wed, 11 Nov 2009 04:35:07 +0000 (23:35 -0500)]
Removed rows length checks in two places, and replaced them with first-column-not-None checks.
Removed an unnecessary conditional from get_block_geometry_as_wkt().

15 years agoRenamed pg2kml to blocks2kml; this is more consistent with the lines2kml script.
Michael Orlitzky [Wed, 11 Nov 2009 00:40:19 +0000 (19:40 -0500)]
Renamed pg2kml to blocks2kml; this is more consistent with the lines2kml script.

15 years agoSet a default usage=None for the CLI.default_option_parser() function.
Michael Orlitzky [Wed, 11 Nov 2009 00:39:00 +0000 (19:39 -0500)]
Set a default usage=None for the CLI.default_option_parser() function.
Updated all of the scripts to use the default_option_parser().

15 years agoAdded the CLI module with a default_option_parser() function.
Michael Orlitzky [Wed, 11 Nov 2009 00:31:13 +0000 (19:31 -0500)]
Added the CLI module with a default_option_parser() function.

15 years agoModified the download_data script to download the TIGER blocks.
Michael Orlitzky [Tue, 10 Nov 2009 15:06:40 +0000 (10:06 -0500)]
Modified the download_data script to download the TIGER blocks.
Removed the blocks download targets from the makefile.

15 years agoAdded the Delaware lines to the download_data script.
Michael Orlitzky [Tue, 10 Nov 2009 05:40:22 +0000 (00:40 -0500)]
Added the Delaware lines to the download_data script.

15 years agoAdded a "Related Projects" heading containing a description of TRAGIS.
Michael Orlitzky [Sat, 7 Nov 2009 17:16:00 +0000 (12:16 -0500)]
Added a "Related Projects" heading containing a description of TRAGIS.

15 years agoReplaced a missing "not".
Michael Orlitzky [Sat, 7 Nov 2009 17:15:20 +0000 (12:15 -0500)]
Replaced a missing "not".

15 years agoAdded Python itself, and Shapely to the software requirements.
Michael Orlitzky [Sat, 7 Nov 2009 17:14:37 +0000 (12:14 -0500)]
Added Python itself, and Shapely to the software requirements.

15 years agoChanged most of the <em> elements to <q>.
Michael Orlitzky [Sat, 7 Nov 2009 17:13:46 +0000 (12:13 -0500)]
Changed most of the <em> elements to <q>.

15 years agoAdded styles for blockquotes and regular quotes (<q> elements).
Michael Orlitzky [Sat, 7 Nov 2009 17:11:50 +0000 (12:11 -0500)]
Added styles for blockquotes and regular quotes (<q> elements).
Changed some of the reset styles to use 'inherit' since we don't have to worry about Internet Explorer screwing them up.

15 years agoAdded the first draft of drag_wkt_along_kml_path, most of which is basically cut...
Michael Orlitzky [Sun, 25 Oct 2009 22:23:53 +0000 (18:23 -0400)]
Added the first draft of drag_wkt_along_kml_path, most of which is basically cut-and-pasted from the wkt2kml script.

15 years agoAdded the KML tests to the test script.
Michael Orlitzky [Sun, 25 Oct 2009 22:23:36 +0000 (18:23 -0400)]
Added the KML tests to the test script.

15 years agoAdded the KMLTest module containing the LineString tests.
Michael Orlitzky [Sun, 25 Oct 2009 22:12:34 +0000 (18:12 -0400)]
Added the KMLTest module containing the LineString tests.

15 years agoMerged branch 'master' with the to_kml() fix.
Michael Orlitzky [Sun, 25 Oct 2009 22:11:24 +0000 (18:11 -0400)]
Merged branch 'master' with the to_kml() fix.

15 years agoAdded the LineString class.
Michael Orlitzky [Sun, 25 Oct 2009 22:09:44 +0000 (18:09 -0400)]
Added the LineString class.

15 years agoModified the Geometry.Polygon.from_wkt() method to return a Geometry.Polygon.
Michael Orlitzky [Sun, 25 Oct 2009 22:08:03 +0000 (18:08 -0400)]
Modified the Geometry.Polygon.from_wkt() method to return a Geometry.Polygon.

15 years agoFixed a bug where the Document open/close tags were not output with to_kml().
Michael Orlitzky [Sun, 25 Oct 2009 05:16:28 +0000 (01:16 -0400)]
Fixed a bug where the Document open/close tags were not output with to_kml().

15 years agoUpdated the Geometry tests to use the new operator overloads.
Michael Orlitzky [Sun, 25 Oct 2009 05:14:56 +0000 (01:14 -0400)]
Updated the Geometry tests to use the new operator overloads.

15 years agoChanged the TwoVector add() method to an operator overload.
Michael Orlitzky [Sun, 25 Oct 2009 05:10:30 +0000 (01:10 -0400)]
Changed the TwoVector add() method to an operator overload.
Added an operator overload for TwoVector subtraction.
Added the wkt() method for Polygons.

15 years agoChanged the wording of a comment.
Michael Orlitzky [Sun, 25 Oct 2009 03:35:43 +0000 (23:35 -0400)]
Changed the wording of a comment.

15 years agoAdded the Geometry module and its tests.
Michael Orlitzky [Sat, 24 Oct 2009 23:43:15 +0000 (19:43 -0400)]
Added the Geometry module and its tests.

15 years agoChanged my mind about using the SVN version of Shapely. Now using v1.0.14.
Michael Orlitzky [Wed, 21 Oct 2009 19:26:12 +0000 (15:26 -0400)]
Changed my mind about using the SVN version of Shapely. Now using v1.0.14.

15 years agoAdded the "lib" makefile task.
Michael Orlitzky [Wed, 21 Oct 2009 18:53:44 +0000 (14:53 -0400)]
Added the "lib" makefile task.
Created a README in the "lib" directory explaining the Shapely checkout procedure.

15 years agoAdded a space between the road name and its tlid.
Michael Orlitzky [Wed, 21 Oct 2009 05:42:16 +0000 (01:42 -0400)]
Added a space between the road name and its tlid.
Fixed the query to actually return KML.

15 years agoAdded XML escaping to the KML classes.
Michael Orlitzky [Wed, 21 Oct 2009 05:41:50 +0000 (01:41 -0400)]
Added XML escaping to the KML classes.

15 years agoAdded the lines2kml script.
Michael Orlitzky [Mon, 19 Oct 2009 14:56:26 +0000 (10:56 -0400)]
Added the lines2kml script.

15 years agoUpdated the KML classes with a print_kml() method.
Michael Orlitzky [Mon, 19 Oct 2009 14:17:02 +0000 (10:17 -0400)]
Updated the KML classes with a print_kml() method.
Abstracted the to_kml() and print_kml() methods out in to the KmlObject class for most subclasses.
Modified pg2kml to use the print_kml() method instead of to_kml().

15 years agoAdded a unique index on the tiger_lines "tlid" column.
Michael Orlitzky [Mon, 19 Oct 2009 11:51:40 +0000 (07:51 -0400)]
Added a unique index on the tiger_lines "tlid" column.
Created a quick sed script to filter "BEGIN;" and "END;" from the shp2pgsql output.
Updated the makefile to import the tiger_lines data one row at a time, outside of a transaction.

15 years agoFixed the "all lines" shapefile importing.
Michael Orlitzky [Sun, 11 Oct 2009 18:01:52 +0000 (14:01 -0400)]
Fixed the "all lines" shapefile importing.

15 years agoRemoved a no-longer-correct comment from the makefile.
Michael Orlitzky [Sun, 11 Oct 2009 16:50:21 +0000 (12:50 -0400)]
Removed a no-longer-correct comment from the makefile.

15 years agoAdded the West Virginia county lines files to download_data.
Michael Orlitzky [Sun, 11 Oct 2009 16:49:15 +0000 (12:49 -0400)]
Added the West Virginia county lines files to download_data.

15 years agoAdded the NY and VA county "all lines" files to download_data.
Michael Orlitzky [Thu, 8 Oct 2009 11:38:51 +0000 (07:38 -0400)]
Added the NY and VA county "all lines" files to download_data.

15 years agoAdded a new script, download_data, to aid in the retrieval of the census data.
Michael Orlitzky [Wed, 7 Oct 2009 15:06:05 +0000 (11:06 -0400)]
Added a new script, download_data, to aid in the retrieval of the census data.
Moved the 'all lines' data retrieval out of the makefile and in to the new script (currently: D.C., MD, PA only).

15 years agoAdded the Data module which contains the classes and functions used to download the...
Michael Orlitzky [Wed, 7 Oct 2009 15:02:57 +0000 (11:02 -0400)]
Added the Data module which contains the classes and functions used to download the online census data files.

15 years agoAdded the FileUtils module.
Michael Orlitzky [Wed, 7 Oct 2009 14:58:13 +0000 (10:58 -0400)]
Added the FileUtils module.

15 years agoAdded the MD lines import to the makefile.
Michael Orlitzky [Sun, 4 Oct 2009 17:18:51 +0000 (13:18 -0400)]
Added the MD lines import to the makefile.
Reworked the data target to be more modular.
Made the blocks/lines import via wildcard.
Fixed all of the index/append inconsistencies in the shp2pgsql usage.

15 years agoAdded the D.C. blocks download/import.
Michael Orlitzky [Sun, 4 Oct 2009 05:35:04 +0000 (01:35 -0400)]
Added the D.C. blocks download/import.
Created a tiger_lines task initially handling only the D.C. lines.
Added URLs for all of the MD county all-lines shapefiles.

15 years agoMoved the default initialization for the Census class in to its constructor.
Michael Orlitzky [Sat, 3 Oct 2009 20:51:26 +0000 (16:51 -0400)]
Moved the default initialization for the Census class in to its constructor.
Removed an unnecessary GPS import from CensusTest.

15 years agoAdded a style for the <strong> element.
Michael Orlitzky [Sat, 3 Oct 2009 19:26:29 +0000 (15:26 -0400)]
Added a style for the <strong> element.

15 years agoAdded some possible optimizations to the project overview.
Michael Orlitzky [Sat, 3 Oct 2009 19:20:58 +0000 (15:20 -0400)]
Added some possible optimizations to the project overview.

15 years agoUpdated the makefile to import TIGER block data for MD, VA, PA, and NY.
Michael Orlitzky [Sat, 3 Oct 2009 18:59:51 +0000 (14:59 -0400)]
Updated the makefile to import TIGER block data for MD, VA, PA, and NY.
Removed quotes from some makefile variables.
Redirected psql output to /dev/null.

15 years agoRenamed the "tiger" table to "tiger_blocks" in preparation for the addition of the...
Michael Orlitzky [Sat, 3 Oct 2009 17:26:14 +0000 (13:26 -0400)]
Renamed the "tiger" table to "tiger_blocks" in preparation for the addition of the TIGER road data.
Shortened the name of the "tiger_blkidfp00" method/column to just "blkidfp00".

15 years agoUpdated the makefile to use the 2009 TIGER/Line shapefiles.
Michael Orlitzky [Sat, 3 Oct 2009 04:28:31 +0000 (00:28 -0400)]
Updated the makefile to use the 2009 TIGER/Line shapefiles.

15 years agoSwitched the order of longitude/latitude in a comment.
Michael Orlitzky [Sat, 3 Oct 2009 04:27:51 +0000 (00:27 -0400)]
Switched the order of longitude/latitude in a comment.

15 years agoUpdated the TIGER/Line technical documentation for the 2009 shapefiles.
Michael Orlitzky [Fri, 2 Oct 2009 20:32:27 +0000 (16:32 -0400)]
Updated the TIGER/Line technical documentation for the 2009 shapefiles.

15 years agoAdded default values for the GPS.Coordinates constructor.
Michael Orlitzky [Thu, 1 Oct 2009 13:44:54 +0000 (09:44 -0400)]
Added default values for the GPS.Coordinates constructor.

15 years agoRemoved the math import from GPS.
Michael Orlitzky [Thu, 1 Oct 2009 13:37:14 +0000 (09:37 -0400)]
Removed the math import from GPS.

15 years agoRemoved unused code from the GPS and SummaryFile1 modules.
Michael Orlitzky [Mon, 28 Sep 2009 02:26:56 +0000 (22:26 -0400)]
Removed unused code from the GPS and SummaryFile1 modules.
Removed the tests utilizing that unused code.

15 years agoAdded unit tests for the Census.Database class.
Michael Orlitzky [Mon, 28 Sep 2009 02:24:12 +0000 (22:24 -0400)]
Added unit tests for the Census.Database class.
Created a new method in the Census.Database class for testing.
Updated the test script with the new suite.

15 years agoChanged the default SRID constant to an integer (as opposed to a string).
Michael Orlitzky [Mon, 28 Sep 2009 02:21:22 +0000 (22:21 -0400)]
Changed the default SRID constant to an integer (as opposed to a string).

15 years agoFactored out the contained population code in to the Census.Database class.
Michael Orlitzky [Sun, 27 Sep 2009 21:17:25 +0000 (17:17 -0400)]
Factored out the contained population code in to the Census.Database class.
Modified wkt2pop to use the new Census.Database method.

15 years agoAdded the Census module.
Michael Orlitzky [Sun, 27 Sep 2009 20:56:04 +0000 (16:56 -0400)]
Added the Census module.
Factored out the find_average_population_density database code into the Census.Database class.

15 years agoConverted the SRID argument to type="int".
Michael Orlitzky [Sun, 27 Sep 2009 20:26:43 +0000 (16:26 -0400)]
Converted the SRID argument to type="int".

15 years agoAdded the wkt2pop script.
Michael Orlitzky [Thu, 24 Sep 2009 04:56:46 +0000 (00:56 -0400)]
Added the wkt2pop script.

15 years agoUpdated the docstring for wkt2kml.
Michael Orlitzky [Thu, 24 Sep 2009 04:52:17 +0000 (00:52 -0400)]
Updated the docstring for wkt2kml.

15 years agoMerged the documentation additions and most recent makefile modification (branch...
Michael Orlitzky [Tue, 22 Sep 2009 07:10:34 +0000 (03:10 -0400)]
Merged the documentation additions and most recent makefile modification (branch master).

15 years agoAdded a script which converts an OGC Well-Known Text string to a KML document.
Michael Orlitzky [Tue, 22 Sep 2009 06:50:28 +0000 (02:50 -0400)]
Added a script which converts an OGC Well-Known Text string to a KML document.

15 years agostrip() whitespace from OptionParser descriptions.
Michael Orlitzky [Tue, 22 Sep 2009 06:49:48 +0000 (02:49 -0400)]
strip() whitespace from OptionParser descriptions.

15 years agoAdded a default SRID (4269).
Michael Orlitzky [Tue, 22 Sep 2009 06:45:36 +0000 (02:45 -0400)]
Added a default SRID (4269).

15 years agoUpdated the makefile for PostGIS 1.4.0.
Michael Orlitzky [Tue, 22 Sep 2009 06:44:01 +0000 (02:44 -0400)]
Updated the makefile for PostGIS 1.4.0.
One of the SQL scripts was renamed, and the locations of the PostGIS files are now calculated using pg_config.

15 years agoAdded the pg2kml script which exports PostGIS data to KML format.
Michael Orlitzky [Wed, 16 Sep 2009 03:42:31 +0000 (23:42 -0400)]
Added the pg2kml script which exports PostGIS data to KML format.

15 years agoModified find_avg_population_density to perform a GIS database query.
Michael Orlitzky [Wed, 16 Sep 2009 03:16:43 +0000 (23:16 -0400)]
Modified find_avg_population_density to perform a GIS database query.
Added optparse argument parsing so that database information does not need to be hard-coded in to the script.
Switched the order of (longitude, latitude) in the arguments.

15 years agoAdded the is_negative_float() method to the StringUtils module.
Michael Orlitzky [Wed, 16 Sep 2009 03:05:48 +0000 (23:05 -0400)]
Added the is_negative_float() method to the StringUtils module.

15 years agoAdded the Configuration package with the Defaults module containing some default...
Michael Orlitzky [Wed, 16 Sep 2009 03:04:25 +0000 (23:04 -0400)]
Added the Configuration package with the Defaults module containing some default configuration parameters.

15 years agoUppercased the exit codes per the Python style guidelines.
Michael Orlitzky [Wed, 16 Sep 2009 03:02:54 +0000 (23:02 -0400)]
Uppercased the exit codes per the Python style guidelines.
Added a new exit code, NO_RESULTS.

15 years agoAdded the KML module with enough classes to export our block data as KML.
Michael Orlitzky [Mon, 14 Sep 2009 01:03:20 +0000 (21:03 -0400)]
Added the KML module with enough classes to export our block data as KML.

15 years agoFixed the Block class initialization so that it converts intptlat/intptlon to real...
Michael Orlitzky [Sat, 12 Sep 2009 22:37:45 +0000 (18:37 -0400)]
Fixed the Block class initialization so that it converts intptlat/intptlon to real coordinates.

15 years agoUpdated the comment about the two PostGIS SQL scripts. I actually know what they...
Michael Orlitzky [Sat, 12 Sep 2009 22:33:09 +0000 (18:33 -0400)]
Updated the comment about the two PostGIS SQL scripts. I actually know what they do now.

15 years agoUse a default SRID of 4269 (North America, 1983) when importing the Tiger shapefiles.
Michael Orlitzky [Sat, 12 Sep 2009 22:31:56 +0000 (18:31 -0400)]
Use a default SRID of 4269 (North America, 1983) when importing the Tiger shapefiles.

15 years agoAdded the framework for the PostGIS database integration.
Michael Orlitzky [Sat, 12 Sep 2009 17:41:20 +0000 (13:41 -0400)]
Added the framework for the PostGIS database integration.
Created makefile targets for the Tiger data and census database.
Added comments to the makefile describing the targets.
Created a SQL script to create the sf1_blocks table, and to add indexes on both blkidfp00 columns.
Created a new script, sf1blocks2sql (used by the makefile), which exports the Summary File 1 blocks as SQL "INSERT" statements.

15 years agoAdded tests for the StringUtils module.
Michael Orlitzky [Sat, 12 Sep 2009 16:30:34 +0000 (12:30 -0400)]
Added tests for the StringUtils module.

15 years agoAdded the Tiger data directory to .gitignore.
Michael Orlitzky [Sat, 12 Sep 2009 16:00:28 +0000 (12:00 -0400)]
Added the Tiger data directory to .gitignore.

15 years agoAdded an additional Block class test which checks that all Maryland Geo Blocks have...
Michael Orlitzky [Sat, 12 Sep 2009 15:05:46 +0000 (11:05 -0400)]
Added an additional Block class test which checks that all Maryland Geo Blocks have a population_density() which can be converted to a float.

15 years agoAdded the is_float() method to the StringUtils module.
Michael Orlitzky [Sat, 12 Sep 2009 15:00:27 +0000 (11:00 -0400)]
Added the is_float() method to the StringUtils module.

15 years agoAdded state, county, and tract fields to the Block class.
Michael Orlitzky [Sat, 12 Sep 2009 14:55:14 +0000 (10:55 -0400)]
Added state, county, and tract fields to the Block class.
Renamed the other Block attributes to follow the GeoRecord/Tiger naming convention.
Updated any tests/methods that depended on the Block attribute names.
Throw an error from within the Block class if the "block" attribute is non-integral.

15 years agoAdded the StringUtils module with the is_integer() function.
Michael Orlitzky [Sat, 12 Sep 2009 14:48:25 +0000 (10:48 -0400)]
Added the StringUtils module with the is_integer() function.
Imported StringUtils into SummaryFile1.
Removed the InvalidAreaError, since we need to accept areas of zero.

15 years agoUppercased GeoRecord.MINIMUM_LINE_LENGTH in accordance with PEP 8: http://python...
Michael Orlitzky [Fri, 11 Sep 2009 05:20:47 +0000 (01:20 -0400)]
Uppercased GeoRecord.MINIMUM_LINE_LENGTH in accordance with PEP 8: http://python.org/dev/peps/pep-0008/.

15 years agoAdded the 2008 Tiger/Line shapefile technical documentation.
Michael Orlitzky [Thu, 10 Sep 2009 04:59:19 +0000 (00:59 -0400)]
Added the 2008 Tiger/Line shapefile technical documentation.

15 years agoAdded an InvalidAreaError class.
Michael Orlitzky [Thu, 10 Sep 2009 04:57:39 +0000 (00:57 -0400)]
Added an InvalidAreaError class.
Caused the Block class to raise an InvalidAreaError when its total area is zero.

15 years agoInitial commit.
Michael Orlitzky [Wed, 9 Sep 2009 03:25:03 +0000 (23:25 -0400)]
Initial commit.