# Iterable datasets features

**URL:** <https://discuss.huggingface.co/t/iterable-datasets-features/15576>\
**Category:** 🤗Datasets\
**Created:** [March 10, 2022, 1:51pm UTC](https://discuss.huggingface.co/t/iterable-datasets-features/15576 "2022-03-10T13:51:49Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![frascuchon](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/frascuchon/32/49520_2.png) [@frascuchon](https://discuss.huggingface.co/u/frascuchon)\
**Post date:** [March 10, 2022, 1:51pm UTC](https://discuss.huggingface.co/t/iterable-datasets-features/15576/1 "2022-03-10T13:51:49Z")

</div>

Hi,

I’m new here and I don’t know if question is already resolved.

I’m using dataset with `streaming=True` and I see the dataset features are `None` . It’s a expected behaviour?

```python
import datasets

ds = datasets.load_dataset("app_reviews", split="train", streaming=True)
print(ds.features)

```

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\
**Post date:** [March 11, 2022, 12:34pm UTC](https://discuss.huggingface.co/t/iterable-datasets-features/15576/2 "2022-03-11T12:34:48Z")

</div>

Hi ! When running your code with `datasets` 1.18.4 I get

```auto
{
  'package_name': Value(dtype='string', id=None),
  'review': Value(dtype='string', id=None),
  'date': Value(dtype='string', id=None),
  'star': Value(dtype='int8', id=None)
}

```

However it can sometimes happen that the features are `None` when you load a dataset in streaming mode.

For example, if you load some CSV data in streaming mode, then you actually need to start streaming some data in order to infer the types are going to have each column. `load_dataset` currently doesn’t prefetch the data to infer the types in streaming mode (though we’re discussing it here: [IterableDataset columns and feature types · Issue #3888 · huggingface/datasets · GitHub](https://github.com/huggingface/datasets/issues/3888))

Prefetching can take a few seconds and maybe more if your dataset has lots of NaNs, because in this case you would even have to stream the dataset until you find a non-null example in order to infer the feature types.

I hope that answers the question, and let me know if you have an opinion regarding prefetching to infer the feature types in streaming mode 🙂

---

<div class="post-metadata">

**Author:** ![frascuchon](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/frascuchon/32/49520_2.png) [@frascuchon](https://discuss.huggingface.co/u/frascuchon)\
**Post date:** [March 11, 2022, 6:10pm UTC](https://discuss.huggingface.co/t/iterable-datasets-features/15576/3 "2022-03-11T18:10:47Z")

</div>

Thanks for the response. I’ve tested with a new clean environment and it works.

I understand the need to start streaming data to infer the types, and I think it fits better. Maybe, it could be nice showing a warning message with some info about how to infer the features when they are still `None` for an `IterableDataset`

Again, thanks for your help!

---

<div class="post-metadata">

**Author:** ![log0](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/log0/32/4970_2.png) [@log0](https://discuss.huggingface.co/u/log0)\
**Post date:** [September 7, 2022, 7:19pm UTC](https://discuss.huggingface.co/t/iterable-datasets-features/15576/4 "2022-09-07T19:19:26Z")

</div>

Hello, I have the same problem I think.

> data = load\_dataset(“json”, data\_files=dataset, split=“train”, streaming=True)  
> data.cast\_column(“audio”, datasets.features.Audio(sampling\_rate=16000))

fails with

> File “test\_predict.py”, line 104, in main  
> data.cast\_column(“audio”, datasets.features.Audio(sampling\_rate=16000))  
> File “/home/ugo/miniconda3/envs/audio/lib/python3.8/site-packages/datasets/iterable\_dataset.py”, line 1191, in cast\_column  
> info.features[column] = feature  
> TypeError: ‘NoneType’ object does not support item assignment

@lhoestq Could you describe what you mean by prefetching data ?

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\
**Post date:** [September 8, 2022, 12:32pm UTC](https://discuss.huggingface.co/t/iterable-datasets-features/15576/5 "2022-09-08T12:32:47Z")

</div>

Hi ! You can fix this by passing the `features=` argument to `load_dataset` with the type of all the columns

What I mean by prefetching is that you need to download the first examples of the dataset to know what type each column is

---

<div class="post-metadata">

**Author:** ![log0](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/log0/32/4970_2.png) [@log0](https://discuss.huggingface.co/u/log0)\
**Post date:** [September 8, 2022, 5:04pm UTC](https://discuss.huggingface.co/t/iterable-datasets-features/15576/6 "2022-09-08T17:04:46Z")

</div>

Ok, Thanks ! makes sense.

For other people in the same case here is what I did to build the feature object. I loaded the same dataset (smaller) without streaming=True, then printed dataset.features and copy pasted it in the initialization of a new Feature() object that I passed to load\_dataset with streaming=True. Something like this

```auto
print(smalldataset.features) # {'path': Value(dtype='string', id=None), ...}
features = Features({'path': Value(dtype='string', id=None), ...})
dataset = load_dataset(... , features=features, streaming=True)

```