Test with the latest version of dask

[maximlt]

No description provided.

[hoxbro]

Failing test(s) looks like an ordering issue:

import dask.dataframe as dd
import holoviews as hv
import pandas as pd
import hvplot.dask

class S: ...
self = S()
    
self.df = pd.DataFrame([[1, 2], [3, 4], [5, 6]], columns=['x', 'y'])
self.df = dd.from_pandas(self.df, npartitions=2)
kind = 'scatter'
element = hv.Scatter

plot = self.df.hvplot(x='index', y='y', by='x', kind=kind)
obj = hv.NdOverlay(
    {
        1: element(self.df[self.df.x == 1], 'index', 'y'),
        3: element(self.df[self.df.x == 3], 'index', 'y'),
        5: element(self.df[self.df.x == 5], 'index', 'y'),
    },
    'x',
)

[hoxbro]

Changing sort=True in this line make it sorted, though likely not the correct fix, but likely a fix is around these lines.

hvplot/hvplot/converter.py

Line 2077 in 9700bec

chart = chart.layout() if self.subplots else chart.overlay(sort=False)

[maximlt]

I think something changed in HoloViews. This script runs fine with dask 2024.4.1 (dask-expr 1.0.11 installed but query-planning disabled) but the last line fails with dask 2025.1.0 as the order of the keys is different.

from packaging.version import Version
import dask
if Version(dask.__version__).release < (2025, 1, 0):
    # Important that this is done before before other imports
    dask.config.set({"dataframe.query-planning": False})

import pandas as pd
import dask.dataframe as dd

import holoviews as hv
hv.extension('bokeh')

df = pd.DataFrame([[1, 2], [3, 4], [5, 6]], columns=['x', 'y'])
ddf = dd.from_pandas(df, npartitions=2)

print(ddf.compute())

expected_ddf = hv.NdOverlay(
    {
        1: hv.Scatter(ddf[ddf.x == 1], 'index', 'y'),
        3: hv.Scatter(ddf[ddf.x == 3], 'index', 'y'),
        5: hv.Scatter(ddf[ddf.x == 5], 'index', 'y'),
    },
    'x',
)
actual_ddf = hv.Dataset(ddf, ['x', 'index'], ['y']).to(hv.Scatter, ['index'], ['y'], ['x']).overlay(sort=False)

print(f'{expected_ddf=}')
print(f'{expected_ddf.keys()=}')

print(f'{actual_ddf=}')
print(f'{actual_ddf.keys()=}')

from holoviews.element.comparison import Comparison

Comparison().compare_ndoverlays(expected_ddf, actual_ddf)

[hoxbro]

I think something changed in HoloViews. This script runs fine with dask 2024.4.1 (dask-expr 1.0.11 installed but query-planning disabled) but the last line fails with dask 2025.1.0 as the order of the keys is different.

A better comparison would be with query-planning enabled. Otherwise, the difference could be from the different dask implementations.

[maximlt]

What changed between the different dask dataframe implementations is the output of ddf['x'].unique().compute().

[maximlt]

Here https://github.com/holoviz/holoviews/blob/d12d3dfcb954937392967688c17eb30d00c94e73/holoviews/core/data/dask.py#L214C1-L214C72