# Getting Value Error while using datasets\_interleave\_datasets method **URL:** \ **Category:** Beginners\ **Created:** [December 21, 2022, 2:19pm UTC](https://discuss.huggingface.co/t/getting-value-error-while-using-datasets-interleave-datasets-method/28227 "2022-12-21T14:19:05Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![vsrinivas](https://avatars.discourse-cdn.com/v4/letter/v/3d9bf3/32.png) [@vsrinivas](https://discuss.huggingface.co/u/vsrinivas)\ **Post date:** [December 21, 2022, 2:19pm UTC](https://discuss.huggingface.co/t/getting-value-error-while-using-datasets-interleave-datasets-method/28227/1 "2022-12-21T14:19:06Z")
Hi, I am following the Hugging Face course. In chapter.5, the Datasets Library, under the topic “Big data? Datasets to the rescue” topic, I am trying to execute the below code in Colab notebook as detailed in the course. ```auto combined_dataset = interleave_datasets([pubmed_dataset_streamed, law_dataset_streamed]) list(islice(combined_dataset, 2)) ``` In response, I am getting the below error, for which I am unable to find a solution anywhere on the web including Github and Stackoverflow. Appreciate if someone can look into and let me know the resolution. Thank you. ```auto --------------------------------------------------------------------------- ValueError Traceback (most recent call last) in 2 from datasets import interleave_datasets 3 ----> 4 combined_dataset = interleave_datasets([pubmed_dataset_streamed, law_dataset_streamed]) 5 list(islice(combined_dataset, 2)) 2 frames /usr/local/lib/python3.8/dist-packages/datasets/features/features.py in _check_if_features_can_be_aligned(features_list) 2052 for k, v in features.items(): 2053 if not (isinstance(v, Value) and v.dtype == "null") and name2feature[k] != v: -> 2054 raise ValueError( 2055 f'The features can\'t be aligned because the key {k} of features {features} has unexpected type - {v} (expected either {name2feature[k]} or Value("null").' 2056 ) ValueError: The features can't be aligned because the key meta of features {'meta': {'case_ID': Value(dtype='string', id=None), 'case_jurisdiction': Value(dtype='string', id=None), 'date_created': Value(dtype='string', id=None)}, 'text': Value(dtype='string', id=None)} has unexpected type - {'case_ID': Value(dtype='string', id=None), 'case_jurisdiction': Value(dtype='string', id=None), 'date_created': Value(dtype='string', id=None)} (expected either {'pmid': Value(dtype='int64', id=None), 'language': Value(dtype='string', id=None)} or Value("null"). ``` ---
**Author:** ![mariosasko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mariosasko/32/31548_2.png) [@mariosasko](https://discuss.huggingface.co/u/mariosasko)\ **Post date:** [December 21, 2022, 2:33pm UTC](https://discuss.huggingface.co/t/getting-value-error-while-using-datasets-interleave-datasets-method/28227/2 "2022-12-21T14:33:32Z")
Hi! This error means that the features of the interleaved datasets are not the same, and they have to be for interleaving. To fix it, make sure `pubmed_dataset_streamed.features` is equal to `law_dataset_streamed.features` before the `interleave_datasets` call. ---
**Author:** ![vsrinivas](https://avatars.discourse-cdn.com/v4/letter/v/3d9bf3/32.png) [@vsrinivas](https://discuss.huggingface.co/u/vsrinivas)\ **Post date:** [December 21, 2022, 2:41pm UTC](https://discuss.huggingface.co/t/getting-value-error-while-using-datasets-interleave-datasets-method/28227/3 "2022-12-21T14:41:19Z")
Thanks @mariosasko for the suggestion. Wondering why this is not mentioned in the course! Nevertheless, how do I equate the features in both the datasets; can you give me the syntax? Thank you. ---
**Author:** ![mariosasko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mariosasko/32/31548_2.png) [@mariosasko](https://discuss.huggingface.co/u/mariosasko)\ **Post date:** [December 21, 2022, 3:13pm UTC](https://discuss.huggingface.co/t/getting-value-error-while-using-datasets-interleave-datasets-method/28227/4 "2022-12-21T15:13:10Z")
You can drop the `meta` column (not needed for training) to equate them: ```python combined_dataset = interleave_datasets( [ pubmed_dataset_streamed.remove_columns("meta"), law_dataset_streamed.remove_columns("meta") ] ) ``` (Please make sure you are using the latest release of `datasets` (2.8.0) before running this; check with `import datasets; print(datasets. __version__ )`) ---
**Author:** ![vsrinivas](https://avatars.discourse-cdn.com/v4/letter/v/3d9bf3/32.png) [@vsrinivas](https://discuss.huggingface.co/u/vsrinivas)\ **Post date:** [December 23, 2022, 8:23am UTC](https://discuss.huggingface.co/t/getting-value-error-while-using-datasets-interleave-datasets-method/28227/5 "2022-12-23T08:23:14Z")
@mariosako. I am sorry, I am unable to try your suggestion because now I am encountering a value error while loading the dataset using the streaming option itself. ---
**Author:** ![mariosasko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mariosasko/32/31548_2.png) [@mariosasko](https://discuss.huggingface.co/u/mariosasko)\ **Post date:** [December 23, 2022, 1:45pm UTC](https://discuss.huggingface.co/t/getting-value-error-while-using-datasets-interleave-datasets-method/28227/6 "2022-12-23T13:45:16Z")
Linking the issue: [Getting Value Error while loading a dataset.. · Issue #5388 · huggingface/datasets · GitHub](https://github.com/huggingface/datasets/issues/5388) ---
**Author:** ![Shamik](https://avatars.discourse-cdn.com/v4/letter/s/ea5d25/32.png) [@Shamik](https://discuss.huggingface.co/u/Shamik)\ **Post date:** [January 17, 2023, 6:25am UTC](https://discuss.huggingface.co/t/getting-value-error-while-using-datasets-interleave-datasets-method/28227/9 "2023-01-17T06:25:34Z")
This does indeed work! However, i am interested to know why `law_dataset_streamed.features` doesn’t return anything. I am instantiating the the `law_dataset_streamed` object as ```python law_dataset_streamed = load_dataset( "json", data_files="https://the-eye.eu/public/AI/pile_preliminary_components/FreeLaw_Opinions.jsonl.zst", split="train", streaming=True, ) ``` If one cannot see the list of the features of an `IterableDataset` then it’s difficult to know which features to drop. Coincidentally, i can see the features of the `pubmed_dataset_streamed.features`. `datasets. __version__ ` = 2.8 Thank you ---
**Author:** ![vsrinivas](https://avatars.discourse-cdn.com/v4/letter/v/3d9bf3/32.png) [@vsrinivas](https://discuss.huggingface.co/u/vsrinivas)\ **Post date:** [January 17, 2023, 12:05pm UTC](https://discuss.huggingface.co/t/getting-value-error-while-using-datasets-interleave-datasets-method/28227/10 "2023-01-17T12:05:16Z")
You can not see the columns of law\_dataset\_streamed by calling .features method is because law\_dataset\_streamed is an iterable dataset object (because of streamed=True parameter in the load\_dataset() statement) and to explore the elements of an Iterable Dataset we need to iterate over it. We can access the columns of our streamed dataset as follows as the generator yields a dictionary: > next(iter(pubmed\_dataset\_streamed)).keys() > dict\_keys([‘meta’, ‘text’]) ---
**Author:** ![vsrinivas](https://avatars.discourse-cdn.com/v4/letter/v/3d9bf3/32.png) [@vsrinivas](https://discuss.huggingface.co/u/vsrinivas)\ **Post date:** [January 17, 2023, 12:08pm UTC](https://discuss.huggingface.co/t/getting-value-error-while-using-datasets-interleave-datasets-method/28227/11 "2023-01-17T12:08:26Z")
This helps and solves the problem. However, it looks like we shall first install zstandard before importing datasets library for everything to work correctly. ---
**Author:** ![Shamik](https://avatars.discourse-cdn.com/v4/letter/s/ea5d25/32.png) [@Shamik](https://discuss.huggingface.co/u/Shamik)\ **Post date:** [January 17, 2023, 12:20pm UTC](https://discuss.huggingface.co/t/getting-value-error-while-using-datasets-interleave-datasets-method/28227/12 "2023-01-17T12:20:59Z")
I can indeed see the `features` of the `IterableDataset` as i can see it for `pubmed_dataset_streamed.features` so i don’t understand why it shouldn’t work for `law_dataset_streamed` ---
**Author:** ![mariosasko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mariosasko/32/31548_2.png) [@mariosasko](https://discuss.huggingface.co/u/mariosasko)\ **Post date:** [February 6, 2023, 1:55pm UTC](https://discuss.huggingface.co/t/getting-value-error-while-using-datasets-interleave-datasets-method/28227/13 "2023-02-06T13:55:13Z")
Hi @Shamik! The former dataset has a loading script in which we can define its features in advance, but the latter doesn’t since it reads data from arbitrary local/remote files. Assuming all the rows have the same columns, you can call `resolve_features()` on it to fix this (otherwise, you can pass `features` to `load_dataset` to manually specify the features). ---
**Author:** ![Shamik](https://avatars.discourse-cdn.com/v4/letter/s/ea5d25/32.png) [@Shamik](https://discuss.huggingface.co/u/Shamik)\ **Post date:** [April 8, 2023, 6:15am UTC](https://discuss.huggingface.co/t/getting-value-error-while-using-datasets-interleave-datasets-method/28227/14 "2023-04-08T06:15:19Z")
Thanks for your reply. Have created a PR for the same [Interleaving Datasets Bug Fix by Shamik-07 · Pull Request #478 · huggingface/course · GitHub](https://github.com/huggingface/course/pull/478)