CyberGIS Toolkit Home Page

Parallel PySAL

The Parallel PySAL library provides a set of scalable PySAL functions. Currently, two parallel PySAL components implemented using the Multiprocessing python library, i.e., the Fisher-Jenks classification algorithm and the weights calculation function, are integrated in CyberGIS Toolkit.

Download

Publications

If you publish your work in which Parallel PySAL is used, please cite the following publication(s):

  • Rey, S.J., L. Anselin, R. Pahle, X. Kang, and P. Stephens. 2013. “Parallel Optimal Choropleth Map Classification in PySAL.” International Journal of Geographical Information Sciences. DOI:10.1080/13658816.2012.752094.

Examples

Example: a test script that uses different number of threads, classes, and data sizes

import math
import sys
import time
import numpy as np
from fj_refactored import fisher_jenks, fj_generate_sample
 
 
def testfull():
    """
    Tests the fully enumerated Fisher-Jenks implementation
    """
    cores = [1,2,4,16,32]
    classes = [5,6,7]
    data_sizes = [500, 1000, 2500, 5000, 7500, 10000, 12500, 15000, 17500, 20000, 22500, 25000]
 
    for c in cores:
        for d in data_sizes:
            for k in classes:
                data = np.random.ranf(size=d)
                try:
                    t1 = time.time()
                    #wrapped in try since we will blow out RAM at some point
                    classification = fisher_jenks(data, k, c)
                    t2 = time.time()
                    print "Processed {0} data points in {1} classes using {2} cores. Total time: {3}".format(d, k, c, t2-t1)
                    data = None
                except KeyboardInterrupt:
                    print "Aborting"
                    sys.exit(1)
                except:
                    print "FAILURE: {0} data points.".format(d)

NMI Build and Test Status

References