genetic-orange

How to use rangeExtractor properly for horizontal column virtualization with pinned or sticky column

I have got column and row virtualization working pretty well using TanStack Virtual v3 beta.31. The only remaining issue I cannot figure out is how to properly use the rangeExtractor function properly to not lose pinned columns as the user scrolls. Here is my live example: https://www.material-react-table.dev/?path=/story/features-virtualization--max-virtualization Here is my code for the virtualizer:

const columnVirtualizer = useVirtualizer({
  count: table.getRowModel().rows[0].getVisibleCells().length,
  estimateSize: () => averageColumnWidth,
  getScrollElement: () => tableContainerRef.current,
  horizontal: true,
  measureElement: (element) => element?.getBoundingClientRect().width,
  overscan: 3,
  rangeExtractor: useCallback(
    (range) =>
      [
        ...new Set([...pinnedColumnIndexes, ...defaultRangeExtractor(range)]),
      ].sort((a, b) => a - b),
    [pinnedColumnIndexes],
  ),
});

const columnVirtualizer = useVirtualizer({
  count: table.getRowModel().rows[0].getVisibleCells().length,
  estimateSize: () => averageColumnWidth,
  getScrollElement: () => tableContainerRef.current,
  horizontal: true,
  measureElement: (element) => element?.getBoundingClientRect().width,
  overscan: 3,
  rangeExtractor: useCallback(
    (range) =>
      [
        ...new Set([...pinnedColumnIndexes, ...defaultRangeExtractor(range)]),
      ].sort((a, b) => a - b),
    [pinnedColumnIndexes],
  ),
});

The problem is that as soon as I add a new number into the range with pinnedColumnIndexes, scrolling horizontally will no longer bring in new columns. It is like the virtualizer does not adjust anymore. But it works perfectly fine without any pinned columns. Also the sticky example on the docs seems to not have the correct dependencies installed? https://tanstack.com/virtual/v3/docs/examples/react/sticky

React Virtual Sticky Example | TanStack Virtual Docs

An example showing how to implement Sticky in React Virtual

26 Replies

sunny-green•3y ago

I have the range extractor working for sticky columns with a tanstack table, (and also noticed the sticky example dependencies were wrong, but was able to clone the repo and get it working locally). I'll try and take a look later, and see if I can figure out what's going on here. Are you trying to do the 'active sticky' thing like in the example, or just (e.g.) all left pinned columns permanently sticky?

genetic-orangeOP•3y ago

The virtualization seems to break if any extra indexes are added to the returned array in rangeExtractor. The active sticky thing in the example was just a dynamic value being added. Not really different than what I'm doing here I don't think.

genetic-orangeOP•3y ago

And to be clear what the issue is, when a column is pinned and the user starts scrolling, this happens:

sunny-green•3y ago

I guess from the screenshot that there should be column 18, 19 etc?

genetic-orangeOP•3y ago

yeah there are supposed to be 500 columns

sunny-green•3y ago

Understood. I definitely have this working so will try to help if I can Am not at my desk just now

genetic-orangeOP•3y ago

and to be clear, when there are no pinned columns at all (normal extraction) it all works perfectly fine

genetic-orangeOP•3y ago

depending on how much you'd want to dig into my issue, this is some of the full source https://github.com/KevinVandy/material-react-table/blob/main/packages/material-react-table/src/table/MRT_Table.tsx I'll see if I can re-create the sticky example for horizontal columns in a sandbox sometime today too

sunny-green•3y ago

Sandbox would be helpful so I could play around with it more easily. Comparing to my own (working) code, there are a few differences but it's hard to say that any of them are obviously to blame. 1. I haven't wrapped the range extractor in useCallback 2. I don't define the sticky columns in the body of the react component, I calculate which indexes are sticky within the range selector callback (I just put the table itself into the closure and read from there). Code snippet below. 3. I don't sort the range array after creation (should I?? it didn't occur to me to check if creating a Set would preserve order) 4. I have not specified a measureElement behaviour (the default is working fine) 5. I have getItemKey defined to get the id of the column.

useVirtualizer({
    count: table.getVisibleLeafColumns().length,
    getScrollElement: () => parentRef.current,
    estimateSize,
    overscan: 3,
    rangeExtractor: (range) => {
      const columns = table.getVisibleLeafColumns()
      const stickyIndices = getStickyIndices(columns)

      return [...new Set([...stickyIndices, ...defaultRangeExtractor(range)])]
    },
    getItemKey: (index) => table.getVisibleLeafColumns()[index].id,
    horizontal: true,
  })

useVirtualizer({
    count: table.getVisibleLeafColumns().length,
    getScrollElement: () => parentRef.current,
    estimateSize,
    overscan: 3,
    rangeExtractor: (range) => {
      const columns = table.getVisibleLeafColumns()
      const stickyIndices = getStickyIndices(columns)

      return [...new Set([...stickyIndices, ...defaultRangeExtractor(range)])]
    },
    getItemKey: (index) => table.getVisibleLeafColumns()[index].id,
    horizontal: true,
  })

For me, I'm not currently using the pinned columns to determine which are sticky, (but it shouldn't matter). That's under consideration, but for now I'm referring to a custom property we have set up on columnDef.meta which defines which 'type' of column it is. @KevinVandy the only one of these differences which I can see potentially being important is getItemKey. The rest seem unlikely to be impactful. I suspect that if you don't specify getItemKey then the virtualiser will use the item index as a key, which (when combined with a custom range extractor) could lead to mixing up which virtual item relates to which 'real' item. @piecyk would be able to confirm I think. If that doesn't work and you share a sandbox I am happy to poke around a bit.

genetic-orangeOP•3y ago

Your thorough reply is very much appreciated. I didn't get any different results by trying this stuff out, so I'll have to keep digging

sunny-green•3y ago

No worries. I have been thinking about opening a PR with a virtualized columns example for tanstack table Maybe that would help (with sticky columns and headers, that is)

genetic-orangeOP•3y ago

actually I think I've found the problem, but no solution yet... instead of using absolute positioning with a translateX on cells, I'm using this pattern to make the left and right spacing at the left and right of the tables

  let virtualPaddingLeft: number | undefined;
  let virtualPaddingRight: number | undefined;

  if (columnVirtualizer && virtualColumns?.length) {
    virtualPaddingLeft = virtualColumns[0]?.start ?? 0;
    virtualPaddingRight =
      columnVirtualizer.getTotalSize() -
      (virtualColumns[virtualColumns.length - 1]
        ?.end ?? 0);
  }

  let virtualPaddingLeft: number | undefined;
  let virtualPaddingRight: number | undefined;

  if (columnVirtualizer && virtualColumns?.length) {
    virtualPaddingLeft = virtualColumns[0]?.start ?? 0;
    virtualPaddingRight =
      columnVirtualizer.getTotalSize() -
      (virtualColumns[virtualColumns.length - 1]
        ?.end ?? 0);
  }

apparently, I need to change the virtualColumns[0].start to virtualColumns[pinnedIndexes.length].start and it almost works I'm actually so close

  const [leftPinnedIndexes, rightPinnedIndexes] = useMemo(
    () =>
      enableColumnVirtualization && enablePinning
        ? [
            table.getLeftLeafColumns().map((c) => c.getPinnedIndex()),
            table
              .getRightLeafColumns()
              .map(
                (c) =>
                  table.getVisibleLeafColumns().length - c.getPinnedIndex() - 1,
              ),
          ]
        : [[], []],
    [columnPinning, enableColumnVirtualization, enablePinning],
  );

  const columnVirtualizer:
    | Virtualizer<HTMLDivElement, HTMLTableCellElement>
    | undefined = enableColumnVirtualization
    ? useVirtualizer({
        count: table.getVisibleLeafColumns().length,
        estimateSize: () => averageColumnWidth,
        getScrollElement: () => tableContainerRef.current,
        horizontal: true,
        measureElement: (element) => element?.getBoundingClientRect().width,
        overscan: 3,
        rangeExtractor: useCallback(
          (range: Range) => [
            ...new Set([
              ...leftPinnedIndexes,
              ...defaultRangeExtractor(range),
              ...rightPinnedIndexes,
            ]),
          ],
          [leftPinnedIndexes, rightPinnedIndexes],
        ),
        ...vProps,
      })
    : undefined;

  const virtualColumns = columnVirtualizer
    ? columnVirtualizer.getVirtualItems()
    : undefined;

  let virtualPaddingLeft: number | undefined;
  let virtualPaddingRight: number | undefined;

  if (columnVirtualizer && virtualColumns?.length) {
    virtualPaddingLeft = virtualColumns[leftPinnedIndexes.length]?.start ?? 0;
    virtualPaddingRight =
      columnVirtualizer.getTotalSize() -
      (virtualColumns[virtualColumns.length - 1 - rightPinnedIndexes.length]
        ?.end ?? 0);
  }

  const [leftPinnedIndexes, rightPinnedIndexes] = useMemo(
    () =>
      enableColumnVirtualization && enablePinning
        ? [
            table.getLeftLeafColumns().map((c) => c.getPinnedIndex()),
            table
              .getRightLeafColumns()
              .map(
                (c) =>
                  table.getVisibleLeafColumns().length - c.getPinnedIndex() - 1,
              ),
          ]
        : [[], []],
    [columnPinning, enableColumnVirtualization, enablePinning],
  );

  const columnVirtualizer:
    | Virtualizer<HTMLDivElement, HTMLTableCellElement>
    | undefined = enableColumnVirtualization
    ? useVirtualizer({
        count: table.getVisibleLeafColumns().length,
        estimateSize: () => averageColumnWidth,
        getScrollElement: () => tableContainerRef.current,
        horizontal: true,
        measureElement: (element) => element?.getBoundingClientRect().width,
        overscan: 3,
        rangeExtractor: useCallback(
          (range: Range) => [
            ...new Set([
              ...leftPinnedIndexes,
              ...defaultRangeExtractor(range),
              ...rightPinnedIndexes,
            ]),
          ],
          [leftPinnedIndexes, rightPinnedIndexes],
        ),
        ...vProps,
      })
    : undefined;

  const virtualColumns = columnVirtualizer
    ? columnVirtualizer.getVirtualItems()
    : undefined;

  let virtualPaddingLeft: number | undefined;
  let virtualPaddingRight: number | undefined;

  if (columnVirtualizer && virtualColumns?.length) {
    virtualPaddingLeft = virtualColumns[leftPinnedIndexes.length]?.start ?? 0;
    virtualPaddingRight =
      columnVirtualizer.getTotalSize() -
      (virtualColumns[virtualColumns.length - 1 - rightPinnedIndexes.length]
        ?.end ?? 0);
  }

genetic-orangeOP•3y ago

this fixes it unless I'm all the way scrolled at either the beginning or the end of the table, but the middle scroll of the table is fixed.

genetic-orangeOP•3y ago

and some negative margin-left css with the column size for pinned columns now fixes it 100%. I probably have some code that would cause most to throw up, but it works lol

sunny-green•3y ago

Oh, I forgot all about that. I vaguely remember doing the same thing to calculate the left and right spacing. I think maybe I also added the width of the sticky columns at that point though, rather than negative margin. Can you share some snippets of the row? I guess you are just adding a td with some padding, and using position: sticky on the relevant columns. I have a janky behaviour where if you scroll very fast to the left, the sticky columns don't keep up with the scroll. Once you come to a stop everything recalculates properly and they do show up in the correct place, it's just that they get scrolled off-screen temporarily while the scroll is underway. It only happens when scrolling left (for left pinned columns, that is). Do you have the same problem? I also wonder if you had the border problem causing tiny gaps to appear between sticky cells. I have a nasty workaround by setting border-collapse:separate and then using inset box shadow instead of real borders (because borders add to the size of the cell and make it a little more awkward to correctly calculate some of the positions). Even with box-sizing: border-box applied to the cells, border was still additive for some reason. Very frustrating!

genetic-orangeOP•3y ago

I have a janky behaviour where if you scroll very fast to the left, the sticky columns don't keep up with the scroll.

I actually have the same The negative margin is actually not perfect, and probably should be replaced. https://www.material-react-table.dev/?path=/story/features-virtualization--max-virtualization The code snippet above is my most recent, though I didn't include all the CSS

sunny-green•3y ago

Interesting that you have the same. I do have some ideas for an alternative approach but it is a bit hacky and requires non-trivial extra effort for accessibility and layout so I haven't tried implementing it yet.

exotic-emerald•3y ago

@KevinVandy did you solve it? What's the story with negative margin 🤔

sunny-green•3y ago

@KevinVandy have you experienced any perf issues with large number of columns? I was having table renders around 120ms even with virtualization. When I took a look with a profiler, it was spending 80ms of that time in getIsVisible for the columns, which seems odd. (number of columns wasn't even that crazy - just a few hundred)

genetic-orangeOP•3y ago

column.getIsVisible() is a really simple method internally that just looks directly at the state

genetic-orangeOP•3y ago

I have not measured the performance myself, but it seems somewhat smooth for me with 500 columns and 10k rows

sunny-green•3y ago

Yeah, I didn't check the source but I assumed it was a simple lookup. I'll dig a little more into it.

unwilling-turquoise•2y ago

To virtualize 100 columns, should I use rangeExtractor? I try to optimize table, based on https://stackblitz.com/edit/github-ei2kl4-mlj26c?file=src%2Fmain.tsx example, but it seems that table renders all columns without virtualization( Do you have working table example with many virtulized columns?

unwilling-turquoise•2y ago

exotic-emerald•2y ago

To virtualize 100 columns, should I use rangeExtractor?

No, rangeExtractor is responsible only what elements should be rendered in the end. @delonge2699 fist issue there that wrong scroll element is used in columnVirtualizer

unwilling-turquoise•2y ago

I tried to use containerRef as scroll element in columnVirtualizer, but that didn't help, after that I added rangeExtractor to render only visible columns, but unsuccessfuly, horizontal scroll worked incorrectly

Gaming

Programming

How to use rangeExtractor properly for horizontal column virtualization with pinned or sticky column

Did you find this page helpful?