# dataiter

The following functions are shorthand helpers for use in conjunction
with [`DataFrame.aggregate()`](data-frame.md#dataiter.DataFrame.aggregate), see the guide on [aggregation](aggregation.md) for details.

[`all()`](#dataiter.all)
[`any()`](#dataiter.any)
[`count()`](#dataiter.count)
[`count_unique()`](#dataiter.count_unique)
[`first()`](#dataiter.first)
[`last()`](#dataiter.last)
[`max()`](#dataiter.max)
[`mean()`](#dataiter.mean)
[`median()`](#dataiter.median)
[`min()`](#dataiter.min)
[`mode()`](#dataiter.mode)
[`nth()`](#dataiter.nth)
[`quantile()`](#dataiter.quantile)
[`std()`](#dataiter.std)
[`sum()`](#dataiter.sum)
[`var()`](#dataiter.var)

The following read functions are convenience aliases to the correspoding
methods of the classes generally most suitable for the particular file
type, i.e. [`DataFrame`](data-frame.md#dataiter.DataFrame) for CSV, NPZ and Parquet,
[`GeoJSON`](geojson.md#dataiter.GeoJSON) for GeoJSON and [`ListOfDicts`](list-of-dicts.md#dataiter.ListOfDicts) for JSON.

[`read_csv()`](#dataiter.read_csv)
[`read_geojson()`](#dataiter.read_geojson)
[`read_json()`](#dataiter.read_json)
[`read_npz()`](#dataiter.read_npz)
[`read_parquet()`](#dataiter.read_parquet)

The following constants can be used to customize certain defaults, such as
formatting and limits for printing.

[`dataiter.PRINT_MAX_WIDTH`](#dataiter.PRINT_MAX_WIDTH)
[`dataiter.PRINT_THOUSAND_SEPARATOR`](#dataiter.PRINT_THOUSAND_SEPARATOR)
[`dataiter.PRINT_TRUNCATE_WIDTH`](#dataiter.PRINT_TRUNCATE_WIDTH)
[`dataiter.USE_NUMBA`](#dataiter.USE_NUMBA)
[`dataiter.USE_NUMBA_CACHE`](#dataiter.USE_NUMBA_CACHE)

<a id="module-dataiter"></a>

### dataiter.PRINT_MAX_WIDTH *= 80*

Maximum amount of columns to wrap print output to. This is only a fallback
in case Python’s `shutil.get_terminal_size` fails to detect the width of
your terminal. By default the detected full width is used.

### dataiter.PRINT_THOUSAND_SEPARATOR *= ''*

Thousand separator to use when printing numbers. By default this is blank,
meaning no thousand separators are rendered.

### dataiter.PRINT_TRUNCATE_WIDTH *= 36*

Maximum width to truncate string columns to in [`DataFrame`](data-frame.md#dataiter.DataFrame) print
output. When this is exceeded, strings will be cut and an ellipsis (`…`)
rendered at the cut point.

### dataiter.USE_NUMBA *= False*

`True` to use Numba, if available, to speed up [aggregations](aggregation.md), `False` to only use pure Python code.

### dataiter.USE_NUMBA_CACHE *= True*

`True` to use Numba cache for JIT-compiled [aggregations](aggregation.md), `False` to only keep compiled code in memory for the
duration of the session.

### dataiter.all(x)

Return whether all elements of x evaluate to `True`.

If x is a string, return a function usable with
[`DataFrame.aggregate()`](data-frame.md#dataiter.DataFrame.aggregate) that operates group-wise on column x.

Uses `numpy.all`, see the NumPy documentation for details:
[https://numpy.org/doc/stable/reference/generated/numpy.all.html](https://numpy.org/doc/stable/reference/generated/numpy.all.html)

```pycon
>>> di.all(di.Vector([True, False]))
False
>>> di.all(di.Vector([True, True]))
True
>>> di.all("x")
<function all.<locals>.aggregate at 0x782ac41a3270>
```

### dataiter.any(x)

Return whether any element of x evaluates to `True`.

If x is a string, return a function usable with
[`DataFrame.aggregate()`](data-frame.md#dataiter.DataFrame.aggregate) that operates group-wise on column x.

Uses `numpy.any`, see the NumPy documentation for details:
[https://numpy.org/doc/stable/reference/generated/numpy.any.html](https://numpy.org/doc/stable/reference/generated/numpy.any.html)

```pycon
>>> di.any(di.Vector([False, False]))
False
>>> di.any(di.Vector([True, False]))
True
>>> di.any("x")
<function any.<locals>.aggregate at 0x7141a070f270>
```

### dataiter.count(x='', \*, drop_na=False)

Return the amount of elements in x.

If x is a string, return a function usable with
[`DataFrame.aggregate()`](data-frame.md#dataiter.DataFrame.aggregate) that operates group-wise on column x. Since
all columns in a data frame should have the same amount of elements (i.e.
rows), you can just leave the x argument at its default blank string, which
will give you that row count.

```pycon
>>> di.count(di.Vector([1, 2, 3]))
3
>>> di.count()
<function count.<locals>.aggregate at 0x71088ce17270>
```

### dataiter.count_unique(x, \*, drop_na=False)

Return the amount of unique elements in x.

If x is a string, return a function usable with
[`DataFrame.aggregate()`](data-frame.md#dataiter.DataFrame.aggregate) that operates group-wise on column x.

```pycon
>>> di.count_unique(di.Vector([1, 2, 2, 3, 3, 3]))
3
>>> di.count_unique("x")
<function count_unique.<locals>.aggregate at 0x7e054462b270>
```

### dataiter.first(x, \*, drop_na=False)

Return the first element of x.

If x is a string, return a function usable with
[`DataFrame.aggregate()`](data-frame.md#dataiter.DataFrame.aggregate) that operates group-wise on column x.

```pycon
>>> di.first(di.Vector([1, 2, 3]))
1
>>> di.first("x")
<function nth.<locals>.aggregate at 0x71ba6e2d7270>
```

### dataiter.last(x, \*, drop_na=False)

Return the last element of x.

If x is a string, return a function usable with
[`DataFrame.aggregate()`](data-frame.md#dataiter.DataFrame.aggregate) that operates group-wise on column x.

```pycon
>>> di.last(di.Vector([1, 2, 3]))
3
>>> di.last("x")
<function nth.<locals>.aggregate at 0x7f97ee113270>
```

### dataiter.max(x, \*, drop_na=True)

Return the maximum of elements in x.

If x is a string, return a function usable with
[`DataFrame.aggregate()`](data-frame.md#dataiter.DataFrame.aggregate) that operates group-wise on column x.

```pycon
>>> di.max(di.Vector([4, 5, 6]))
6
>>> di.max("x")
<function max.<locals>.aggregate at 0x756d554af270>
```

### dataiter.mean(x, \*, drop_na=True)

Return the arithmetic mean of x.

If x is a string, return a function usable with
[`DataFrame.aggregate()`](data-frame.md#dataiter.DataFrame.aggregate) that operates group-wise on column x.

Uses `numpy.mean`, see the NumPy documentation for details:
[https://numpy.org/doc/stable/reference/generated/numpy.mean.html](https://numpy.org/doc/stable/reference/generated/numpy.mean.html)

```pycon
>>> di.mean(di.Vector([1, 2, 10]))
4.333333333333333
>>> di.mean("x")
<function mean.<locals>.aggregate at 0x71502afa3270>
```

### dataiter.median(x, \*, drop_na=True)

Return the median of x.

If x is a string, return a function usable with
[`DataFrame.aggregate()`](data-frame.md#dataiter.DataFrame.aggregate) that operates group-wise on column x.

Uses `numpy.median`, see the NumPy documentation for details:
[https://numpy.org/doc/stable/reference/generated/numpy.median.html](https://numpy.org/doc/stable/reference/generated/numpy.median.html)

```pycon
>>> di.median(di.Vector([5, 1, 2]))
2.0
>>> di.median("x")
<function median.<locals>.aggregate at 0x7863e44e3270>
```

### dataiter.min(x, \*, drop_na=True)

Return the minimum of elements in x.

If x is a string, return a function usable with
[`DataFrame.aggregate()`](data-frame.md#dataiter.DataFrame.aggregate) that operates group-wise on column x.

```pycon
>>> di.min(di.Vector([4, 5, 6]))
4
>>> di.min("x")
<function min.<locals>.aggregate at 0x74b14b9e7270>
```

### dataiter.mode(x, \*, drop_na=True)

Return the most common value in x.

If x is a string, return a function usable with
[`DataFrame.aggregate()`](data-frame.md#dataiter.DataFrame.aggregate) that operates group-wise on column x.

```pycon
>>> di.mode(di.Vector([1, 2, 2, 3, 3, 3]))
3
>>> di.mode("x")
<function mode.<locals>.aggregate at 0x74e549d5b270>
```

### dataiter.nth(x, index, \*, drop_na=False)

Return the element of x at index (zero-based).

If x is a string, return a function usable with
[`DataFrame.aggregate()`](data-frame.md#dataiter.DataFrame.aggregate) that operates group-wise on column x.

```pycon
>>> di.nth(di.Vector([1, 2, 3]), 1)
2
>>> di.nth("x", 1)
<function nth.<locals>.aggregate at 0x7d58dca4b110>
```

### dataiter.quantile(x, q, \*, drop_na=True)

Return the qth quantile of x.

If x is a string, return a function usable with
[`DataFrame.aggregate()`](data-frame.md#dataiter.DataFrame.aggregate) that operates group-wise on column x.

Uses `numpy.quantile`, see the NumPy documentation for details:
[https://numpy.org/doc/stable/reference/generated/numpy.quantile.html](https://numpy.org/doc/stable/reference/generated/numpy.quantile.html)

```pycon
>>> di.quantile(di.Vector([1, 5, 6]), 0.5)
5.0
>>> di.quantile("x", 0.5)
<function quantile.<locals>.aggregate at 0x7a5c74a6b110>
```

### dataiter.read_csv(path, \*, encoding='utf-8', sep=',', header=True, columns=[], dtypes={})

Return a new data frame from CSV file path.

Will automatically decompress if path ends in `.bz2|.gz|.xz`.
columns is an optional list of columns to limit to. dtypes is an
optional dict mapping column names to NumPy datatypes.

> #### NOTE
> [`read_csv()`](#dataiter.read_csv) is a convenience alias for [`DataFrame.read_csv()`](data-frame.md#dataiter.DataFrame.read_csv).

### dataiter.read_geojson(path, \*, encoding='utf-8', columns=[], dtypes={}, \*\*kwargs)

Return data from GeoJSON file path.

Will automatically decompress if path ends in `.bz2|.gz|.xz`.
columns is an optional list of columns to limit to. dtypes is an
optional dict mapping column names to NumPy datatypes. kwargs are
passed to `json.load`.

> #### NOTE
> [`read_geojson()`](#dataiter.read_geojson) is a convenience alias for [`GeoJSON.read()`](geojson.md#dataiter.GeoJSON.read).

### dataiter.read_json(path, \*, encoding='utf-8', keys=[], types={}, \*\*kwargs)

Return a new list from JSON file path.

Will automatically decompress if path ends in `.bz2|.gz|.xz`.
keys is an optional list of keys to limit to. types is an optional
dict mapping keys to datatypes. kwargs are passed to `json.load`.

> #### NOTE
> [`read_json()`](#dataiter.read_json) is a convenience alias for [`ListOfDicts.read_json()`](list-of-dicts.md#dataiter.ListOfDicts.read_json).

### dataiter.read_npz(path, \*, allow_pickle=True)

Return a new data frame from NumPy file path.

See numpy.load for an explanation of allow_pickle:
[https://numpy.org/doc/stable/reference/generated/numpy.load.html](https://numpy.org/doc/stable/reference/generated/numpy.load.html)

> #### NOTE
> [`read_npz()`](#dataiter.read_npz) is a convenience alias for [`DataFrame.read_npz()`](data-frame.md#dataiter.DataFrame.read_npz).

### dataiter.read_parquet(path, \*, columns=[], dtypes={})

Return a new data frame from Parquet file path.

columns is an optional list of columns to limit to. dtypes is an
optional dict mapping column names to NumPy datatypes.

> #### NOTE
> [`read_parquet()`](#dataiter.read_parquet) is a convenience alias for [`DataFrame.read_parquet()`](data-frame.md#dataiter.DataFrame.read_parquet).

### dataiter.std(x, \*, ddof=0, drop_na=True)

Return the standard deviation of x.

If x is a string, return a function usable with
[`DataFrame.aggregate()`](data-frame.md#dataiter.DataFrame.aggregate) that operates group-wise on column x.

Uses `numpy.std`, see the NumPy documentation for details:
[https://numpy.org/doc/stable/reference/generated/numpy.std.html](https://numpy.org/doc/stable/reference/generated/numpy.std.html)

```pycon
>>> di.std(di.Vector([3, 6, 7]))
1.699673171197595
>>> di.std("x")
<function std.<locals>.aggregate at 0x708141d6b270>
```

### dataiter.sum(x, \*, drop_na=True)

Return the sum of x.

If x is a string, return a function usable with
[`DataFrame.aggregate()`](data-frame.md#dataiter.DataFrame.aggregate) that operates group-wise on column x.

```pycon
>>> di.sum(di.Vector([1, 2, 3]))
6
>>> di.sum("x")
<function sum.<locals>.aggregate at 0x70f5748c3270>
```

### dataiter.var(x, \*, ddof=0, drop_na=True)

Return the variance of x.

If x is a string, return a function usable with
[`DataFrame.aggregate()`](data-frame.md#dataiter.DataFrame.aggregate) that operates group-wise on column x.

Uses `numpy.var`, see the NumPy documentation for details:
[https://numpy.org/doc/stable/reference/generated/numpy.var.html](https://numpy.org/doc/stable/reference/generated/numpy.var.html)

```pycon
>>> di.var(di.Vector([3, 6, 7]))
2.888888888888889
>>> di.var("x")
<function var.<locals>.aggregate at 0x73c7573d3270>
```