Most real world data though is messy and defining a uniqueness constraint upfront (upon ingestion) is often limiting, so for practical use cases this gets relaxed to a multi-set rather than sparse array model for storage, and uniqueness imposed in some way after the fact (if required).

Ogden and Provo are some of the worst offenders for per household air pollution emissions. Like many western cities they have longish commutes (everywhere) in large cars (trucks / suv's) with a high number of cars / household and almost non-functional public transport system. For the Salt Lake Metro area, per capita carbon emissions doubled between 1980 and 2015 because of increasing sprawl. Air regulations here are spotty for personal vehicles and I'm guessing almost non-existent for commercial vehicles. Oh and the state governments solution to this is to push a publicly subsidized "inland port" that will bring increased truck and rail traffic to the valley. The leaders of these tech companies are starting to point out that terrible air pollution for parts of the year is hurting recruitment so it seems like as the money flows into this sector maybe there will be political will on the state and local side to address some of these issues.

I was referring more to the fact that the cloud vendors can co-design their infrastructure and software to support their database services.

A "thick-client" also doesn't perform well unless that client is located on a node in the same region. I think as with everything it works well in some cases and not well in others.

I like your point about consuming S3 cleverly, it's often difficult to get good out of the box performance from S3 so abstracting that to the degree possible is good for end-users. The cloud vendors though are always one or two steps ahead of companies that build upon their services. AWS Redshift for instance already can pre-index objects stored on S3 to accelerate queries at the storage layer. It's difficult as a third party vendor to compete with that.

https://twitter.com/mikeandersonKSL/status/12577783769258024...

TileDB is a technology for storing and managing enormous volumes of structured data on the cloud. TileDB efficiently stores data generated from variety of domains (genomics, finance, imaging, geospatial, etc) in a novel unified format as sparse or dense multidimensional arrays. Users can efficiently access this data via a growing number of language APIs, interfaces and our upcoming hosted service. TileDB, Inc. has raised $4M in seed funding over the past 18 months.

We are looking for frontend and backend developers to help develop a scalable service to make data storage, management, and computations easier for practicing data and domain scientists. A few features on the roadmap include designing a web admin console for data management, monitoring resources in real time, integration with JupyterLab and handling user profiles, billing, encryption and key management, and scaling / management of serverless compute resources.

We are primarily seeking:

- UI / UX Engineer (Javascript / Vue.js / Golang / K8s)

- Backend engineer (Go, CGo, K8’s)

* TileDB GitHub organization: https://github.com/TileDB-Inc

* TileDB has been featured on HN: https://news.ycombinator.com/item?id=15547749.

* TileDB blog post: http://bit.ly/2SjkeYR

* TileDB talk at PyData 2018: http://bit.ly/2BfbyJ3

Our headquarters are located in Cambridge, MA. The candidates must be US citizens or permanent residents located in the US, or Greek citizens to be located at our subsidiary in Athens, Greece.

Apply today at https://tiledb.workable.com !

The other hidden cost is that these ancient software systems are extremely efficient in terms of cpu / memory utilization compared to modern equivalents so the hardware requirements go up considerably which impacts power / energy efficiency and direct deployment costs for replacement systems. Not saying modernization is not a win in the long term but there are direct and indirect costs with these big transitions.

TileDB is a technology for storing and managing enormous volumes of structured data on the cloud. TileDB efficiently stores data generated from variety of domains (genomics, finance, imaging, geospatial, etc) in a novel unified format as sparse or dense multidimensional arrays. Users can efficiently access this data via a growing number of language APIs, interfaces and our upcoming hosted service. TileDB, Inc. has raised $4M in seed funding over the past 18 months.

We are looking for frontend and backend developers to help develop a scalable service to make data storage, management, and computations easier for practicing data and domain scientists. A few features on the roadmap include designing a web admin console for data management, monitoring resources in real time, integration with JupyterLab and handling user profiles, billing, encryption and key management, and scaling / management of serverless compute resources.

We are primarily seeking:

- UI / UX Engineer (Javascript / Vue.js / Golang / K8s)

- Backend engineer (Go, CGo, K8’s)

TileDB GitHub organization: https://github.com/TileDB-Inc

TileDB has been featured on HN: https://news.ycombinator.com/item?id=15547749.

TileDB recent blog post: http://bit.ly/2SjkeYR

TileDB talk at PyData 2018: http://bit.ly/2BfbyJ3

Our headquarters are located in Cambridge, MA. The candidates must be US citizens or permanent residents located in the US, or Greek citizens to be located at our subsidiary in Athens, Greece.

Apply today at https://tiledb.workable.com!