Package dap :: Module helper
[hide private]
[frames] | no frames]

Module helper

source code

Helper functions.

These are generic functions used mostly for writing plugins.

Functions [hide private]
  constrain(dataset, constraints)
A simple example.
  build(dapvar, new, fields, queries)
  filter_(dapvar, queries)
  slicevar(dapvar, slice_)
  getslice(hyperslab, shape=None)
Parse a hyperslab.
  _getsize(dimslice)
Parse a dimension from a hyperslab.
  buildfilter(queries, vars_)
This function is a filter builder.
  multicomp(function)
Multiple OR comparisons.
  fix_slice(dims, index)
Fix incomplete slices or slices with ellipsis.
  lenslice(slice_)
Return the number of values associated with a slice.
  parse_querystring(query)
Parse a query_string returning the requested variables, dimensions, and CEs.
  escape_dods(dods, pad='')
Escape a DODS response.
  _test()

Variables [hide private]
  __author__ = 'Roberto De Almeida <rob@pydap.org>'

Imports: sys, re, operator, itertools, copy, urllib.unquote, dtypes.Float16, dtypes.ArrayType, dtypes.Float32, dtypes.GridType, dtypes.Int32, dtypes.Int16, dtypes.String, dtypes.BaseType, dtypes.UInt16, dtypes.Int0, dtypes.Byte, dtypes.DatasetType, dtypes.Int8, dtypes.StructureType, dtypes.Float64, dtypes.Url, dtypes.SequenceType, dtypes.Int64, dtypes.UInt64, dtypes.Int, dtypes.Float, dtypes.UInt32, dtypes.Float8, dtypes.Float0, dtypes._basetypes, exceptions.ConstraintExpressionError, lib.isiterable, util.safeeval.expr_eval, util.ordereddict.odict


Function Details [hide private]

constrain(dataset, constraints)

source code 
A simple example. We create a dataset holding three variables:
>>> dataset = DatasetType(name='foo')
>>> dataset['a'] = BaseType(name='a', type='Byte')
>>> dataset['b'] = BaseType(name='b', type='Byte')
>>> dataset['c'] = BaseType(name='c', type='Byte')
Now we give it a CE requesting only the variables ``a`` and ``b``:
>>> dataset2 = constrain(dataset, 'a,b')
>>> print dataset2  #doctest: +ELLIPSIS
{'a': <dap.dtypes.BaseType object at ...>, 'b': <dap.dtypes.BaseType object at ...>}
We can also request the variables in a different order:
>>> dataset2 = constrain(dataset, 'b,a')
>>> print dataset2  #doctest: +ELLIPSIS
{'b': <dap.dtypes.BaseType object at ...>, 'a': <dap.dtypes.BaseType object at ...>}
Another example. A dataset with two structures ``a`` and ``b``:
>>> dataset = DatasetType(name='foo')
>>> dataset['a'] = StructureType(name='a')
>>> dataset['a']['a1'] = BaseType(name='a1', type='Byte')
>>> dataset['b'] = StructureType(name='b')
>>> dataset['b']['b1'] = BaseType(name='b1', type='Byte')
>>> dataset['b']['b2'] = BaseType(name='b2', type='Byte')
If we request the structure ``b`` we should get it complete:
>>> dataset2 = constrain(dataset, 'a.a1,b')
>>> print dataset2  #doctest: +ELLIPSIS
{'a': {'a1': <dap.dtypes.BaseType object at ...>}, 'b': {'b1': <dap.dtypes.BaseType object at ...>, 'b2': <dap.dtypes.BaseType object at ...>}}
>>> dataset2 = constrain(dataset, 'b.b1')
>>> print dataset2  #doctest: +ELLIPSIS
{'b': {'b1': <dap.dtypes.BaseType object at ...>}}
Arrays can be sliced. Here we have a ``(2,3)`` array:
>>> dataset = DatasetType(name='foo')
>>> from numpy import array
>>> data = array([1,2,3,4,5,6])
>>> data.shape = (2,3)
>>> dataset['array'] = ArrayType(data=data, name='array', shape=(2,3), type='Int32')
>>> dataset2 = constrain(dataset, 'array')
>>> from dap.server import SimpleHandler
>>> headers, output = SimpleHandler(dataset).dds()
>>> print ''.join(output)
Dataset {
    Int32 array[2][3];
} foo;
<BLANKLINE>
>>> print dataset2['array'].data
[[1 2 3]
 [4 5 6]]
But we request only part of it:
>>> dataset2 = constrain(dataset, 'array[0:1:1][0:1:1]')
>>> headers, output = SimpleHandler(dataset2).dds()
>>> print ''.join(output)
Dataset {
    Int32 array[2][2];
} foo;
<BLANKLINE>
>>> print dataset2['array'].data
[[1 2]
 [4 5]]
The same is valid for grids:
>>> dataset['grid'] = GridType(name='grid') 
>>> data = array([1,2,3,4,5,6])
>>> data.shape = (2,3)
>>> dataset['grid'].array  = ArrayType(name='grid', data=data, shape=(2,3), dimensions=('x', 'y'))
>>> dataset['grid'].maps['x'] = ArrayType(name='x', data=array([1,2]), shape=(2,))
>>> dataset['grid'].maps['y'] = ArrayType(name='y', data=array([1,2,3]), shape=(3,))
>>> dataset._set_id()
>>> headers, output = SimpleHandler(dataset).dds()
>>> print ''.join(output)
Dataset {
    Int32 array[2][3];
    Grid {
        Array:
            Int32 grid[x = 2][y = 3];
        Maps:
            Int32 x[x = 2];
            Int32 y[y = 3];
    } grid;
} foo;
<BLANKLINE>
>>> dataset2 = constrain(dataset, 'grid[0:1:0][0:1:0]')
>>> headers, output = SimpleHandler(dataset2).dds()
>>> print ''.join(output)
Dataset {
    Grid {
        Array:
            Int32 grid[x = 1][y = 1];
        Maps:
            Int32 x[x = 1];
            Int32 y[y = 1];
    } grid;
} foo;
<BLANKLINE>
>>> headers, output = SimpleHandler(dataset2).ascii()
>>> print ''.join(output)
Dataset {
    Grid {
        Array:
            Int32 grid[x = 1][y = 1];
        Maps:
            Int32 x[x = 1];
            Int32 y[y = 1];
    } grid;
} foo;
---------------------------------------------
grid.grid
[0][0] 1
<BLANKLINE>
grid.x
[0] 1
<BLANKLINE>
grid.y
[0] 1
<BLANKLINE>
<BLANKLINE>
<BLANKLINE>
<BLANKLINE>
Selecting a map from a Grid should return a structure:
>>> dataset3 = constrain(dataset, 'grid.x')
>>> headers, output = SimpleHandler(dataset3).dds()
>>> print ''.join(output)
Dataset {
    Structure {
        Int32 x[x = 2];
    } grid;
} foo;
<BLANKLINE>
Short notation also works:
>>> dataset3 = constrain(dataset, 'x')
>>> headers, output = SimpleHandler(dataset3).dds()
>>> print ''.join(output)
Dataset {
    Structure {
        Int32 x[x = 2];
    } grid;
} foo;
<BLANKLINE>
It also works with Sequences:
>>> dataset = DatasetType(name='foo')
>>> dataset['seq'] = SequenceType(name='seq')
>>> dataset['seq']['a'] = BaseType(name='a')
>>> dataset['seq']['b'] = BaseType(name='b')
>>> dataset['seq']['a'].data = range(5)
>>> dataset['seq']['b'].data = range(5,10)
>>> for i in dataset['seq'].data:
...     print i
(0, 5)
(1, 6)
(2, 7)
(3, 8)
(4, 9)
>>> dataset2 = constrain(dataset, 'seq.a')
>>> for i in dataset2['seq'].data:
...     print i
(0,)
(1,)
(2,)
(3,)
(4,)
>>> dataset2 = constrain(dataset, 'seq.b')
>>> for i in dataset2['seq'].data:
...     print i
(5,)
(6,)
(7,)
(8,)
(9,)
>>> dataset2 = constrain(dataset, 'seq.b,seq.a')
>>> for i in dataset2['seq'].data:
...     print i
(5, 0)
(6, 1)
(7, 2)
(8, 3)
(9, 4)
The function also parses selection expressions. Let's create a dataset with sequential data:
>>> dataset = DatasetType(name='foo')
>>> dataset['seq'] = SequenceType(name='seq')
>>> dataset['seq']['index'] = BaseType(name='index', type='Int32')
>>> dataset['seq']['index'].data = [10, 11, 12, 13]
>>> dataset['seq']['temperature'] = BaseType(name='temperature', type='Float32')
>>> dataset['seq']['temperature'].data = [17.2, 15.1, 15.3, 15.1]
>>> dataset['seq']['site'] = BaseType(name='site', type='String')
>>> dataset['seq']['site'].data = ['Diamond_St', 'Blacktail_Loop', 'Platinum_St', 'Kodiak_Trail']
Here's the data:
>>> for i in dataset['seq'].data:
...     print i
(10, 17.199999999999999, 'Diamond_St')
(11, 15.1, 'Blacktail_Loop')
(12, 15.300000000000001, 'Platinum_St')
(13, 15.1, 'Kodiak_Trail')
Now suppose we only want data where ``index`` is greater than 11:
>>> dataset2 = constrain(dataset, 'seq&seq.index>11')
>>> for i in dataset2['seq'].data:
...     print i
(12, 15.300000000000001, 'Platinum_St')
(13, 15.1, 'Kodiak_Trail')
We can request only a few variables:
>>> dataset2 = constrain(dataset, 'seq.site&seq.index>11')
>>> for i in dataset2['seq'].data:
...     print i
('Platinum_St',)
('Kodiak_Trail',)

build(dapvar, new, fields, queries)

source code 
None

filter_(dapvar, queries)

source code 
None

slicevar(dapvar, slice_)

source code 
None

getslice(hyperslab, shape=None)

source code 

Parse a hyperslab.

Parse a hyperslab to a slice according to variable shape. The hyperslab follows the DAP specification, and ommited dimensions are returned in their entirety.
>>> getslice('[0:1:2][0:1:2]')
(slice(0, 3, 1), slice(0, 3, 1))
>>> getslice('[0:2][0:2]')
(slice(0, 3, 1), slice(0, 3, 1))
>>> getslice('[0][2]')
(slice(0, 1, 1), slice(2, 3, 1))
>>> getslice('[0:1:1]')
(slice(0, 2, 1),)
>>> getslice('[0:2:1]')
(slice(0, 2, 2),)
>>> getslice('')
(slice(None, None, None),)

_getsize(dimslice)

source code 

Parse a dimension from a hyperslab.

Calculates the start, size and step from a DAP formatted hyperslab.
>>> _getsize('0:1:9')
(0, 10, 1)
>>> _getsize('0:2:9')
(0, 10, 2)
>>> _getsize('0')
(0, 1, 1)
>>> _getsize('0:9')
(0, 10, 1)

buildfilter(queries, vars_)

source code 

This function is a filter builder.

Given a list of DAP formatted queries and a list of variable names, this function returns a dynamic filter function to filter rows.

From the example in the DAP specification:
>>> vars_ = ['index', 'temperature', 'site']
>>> data = []
>>> data.append([10, 17.2, 'Diamond_St'])
>>> data.