Ticket #20 (closed feature: fixed)

Opened 10 years ago

Last modified 7 years ago

Efficient support for very large image data

Reported by: curtis Owned by: curtis
Priority: critical Milestone: imagej2-b7-ndim-data
Component: ImgLib2 Version:
Severity: major Keywords:
Cc: Blocked By: #216, #583, #1669, #1872, #1885
Blocking:

Description

Datasets are growing in both size and complexity: we are now seeing acquisition systems capable of producing datasets larger than available memory (e.g., 1024x1024x16-bit x 500 timepoints x 50 focal planes = 49GB). The current version of ImageJ provides a mechanism, the virtual stack, for handling such large datasets one image plane at a time.

With imglib it should also be possible to work with such data using a clever storage strategy, but it would be nice to have at least one concrete example in code demonstrating how this should work.

Change History

comment:1 Changed 9 years ago by aivar

  • Status changed from new to accepted

See #20.

comment:2 Changed 9 years ago by aivar

  • Severity changed from non-issue to major
  • Milestone changed from progress-report to biweekly-2011: Mar-28 to Apr-08

comment:3 Changed 9 years ago by curtis

  • Milestone changed from biweekly-2011: Apr-11 to Apr-22 to biweekly-2011: Jun-06 to Jun-17

This sort of testing makes sense to do as part of the 2.0-beta1 development in June.

comment:4 Changed 8 years ago by curtis

  • Status changed from accepted to closed
  • Resolution set to duplicate

We are working on new ImgLib2 container types for this. See ticket #216.

comment:5 Changed 8 years ago by curtis

  • Type changed from task to story

comment:6 Changed 8 years ago by curtis

  • Status changed from closed to reopened
  • Summary changed from Verify very large image data can be handled efficiently to Efficient support for very large image data
  • Resolution duplicate deleted
  • Blocked By 216 added
  • Milestone changed from biweekly-2011: Jul-18 to Jul-29 to imagej-2.0-beta2

This ticket is now a story describing the end goal of supporting large image data. Ticket #216 (and likely others to be added later) will indicate the actual approaches and tasks used to accomplish this goal.

comment:7 Changed 8 years ago by curtis

  • Blocked By 216 removed

comment:8 Changed 8 years ago by curtis

  • Blocked By 216 added

comment:9 Changed 8 years ago by curtis

  • Blocked By 583 added

comment:10 Changed 8 years ago by curtis

  • Owner changed from aivar to curtis
  • Status changed from reopened to assigned
  • Milestone changed from imagej-2.0.0-beta3 to imagej-2.0.0-beta4

comment:11 Changed 7 years ago by curtis

  • Milestone changed from imagej-2.0.0-beta4 to imagej-2.0.0-beta5

The theme of beta5 will be more flexible data visualization and processing.

comment:12 Changed 7 years ago by bdezonia

Just a note that there are two test implementations of a file backed Img in imlgib2 branches: fileimg (by Tobias) and toy-file-backed-img (by Barry)

comment:13 Changed 7 years ago by bdezonia

  • Blocked By 1669 added

comment:14 Changed 7 years ago by curtis

  • Priority changed from major to critical

comment:15 Changed 7 years ago by dscho

  • Blocked By 1819 added

comment:16 Changed 7 years ago by bdezonia

  • Blocked By 1872 added

comment:17 Changed 7 years ago by bdezonia

  • Blocked By 1885 added

comment:18 Changed 7 years ago by curtis

  • Blocked By 1819 removed

(In #1819) Dscho: Where can I find such data? I'd like to load it into beta7 and take a screenshot for the blog post. (But maybe it's not possible yet without some sort of multi-angle registration feature? If so, we can bump this to beta8.)

I am changing this ticket to block the user story, because it is more about the blog post than any technical coding issue.

comment:19 Changed 7 years ago by curtis

  • Status changed from assigned to closed
  • Resolution set to fixed

Thanks to SCIFIOCellImg, we have initial support for huge image data. And this ticket only had to stay open for three and a half years!

Note: See TracTickets for help on using tickets.