Parallel PySAL

The Parallel PySAL library provides a set of scalable PySAL functions. Currently, two parallel PySAL components implemented using the Multiprocessing python library, i.e., the Fisher-Jenks classification algorithm and the weights calculation function, are integrated in CyberGIS Toolkit.

• Download from CyberGIS Toolkit source package
• Github

If you publish your work in which Parallel PySAL is used, please cite the following publication(s):

• Rey, S.J., L. Anselin, R. Pahle, X. Kang, and P. Stephens. 2013. “Parallel Optimal Choropleth Map Classification in PySAL.” International Journal of Geographical Information Sciences. DOI:10.1080/13658816.2012.752094.

Example: a test script that uses different number of threads, classes, and data sizes

import math
import sys
import time
import numpy as np
from fj_refactored import fisher_jenks, fj_generate_sample
 
 
def testfull():
    """
    Tests the fully enumerated Fisher-Jenks implementation
    """
    cores = [1,2,4,16,32]
    classes = [5,6,7]
    data_sizes = [500, 1000, 2500, 5000, 7500, 10000, 12500, 15000, 17500, 20000, 22500, 25000]
 
    for c in cores:
        for d in data_sizes:
            for k in classes:
                data = np.random.ranf(size=d)
                try:
                    t1 = time.time()
                    #wrapped in try since we will blow out RAM at some point
                    classification = fisher_jenks(data, k, c)
                    t2 = time.time()
                    print "Processed {0} data points in {1} classes using {2} cores. Total time: {3}".format(d, k, c, t2-t1)
                    data = None
                except KeyboardInterrupt:
                    print "Aborting"
                    sys.exit(1)
                except:
                    print "FAILURE: {0} data points.".format(d)

• PySAL