Return

This Week in Databend #89

April 15, 2023 · 4 min read

PsiACE

Stay up to date with the latest weekly developments on Databend!


Databend is a modern cloud data warehouse, serving your massive-scale analytics needs at low cost and complexity. Open source alternative to Snowflake. Also available in the cloud: https://app.databend.com .

What's On In Databend

Stay connected with the latest news about Databend.

Announcing Databend v1.1

Databend v1.1.0 was officially released on April 14, 2023! This release marks the first major update of Databend since version 1.0.

The new version has many new features, including:

  • COPY INTO now has ETL capabilities
  • Full Support for TPC-DS Queries
  • REPLACE INTO
  • Window Functions
  • Integration with Metabase, Redash and Grafana
  • Rust driver
  • AI Functions
  • AskBend - knowledge base Q&A system

If you are interested in learning more, please check out the resources listed below.

Databend now offers additional configurations for AI-related tasks, allowing you the flexibility to choose from various endpoints and models.

This means that it can be easily integrated with other services including Azure OpenAI API. Additionally, the corresponding model can be flexibly configured as needed to balance performance and cost.

| "query"   | "openai_api_base_url"                      | "https://api.openai.com/v1/"     | ""       |
| "query" | "openai_api_completion_model" | "text-embedding-ada-002" | "" |
| "query" | "openai_api_embedding_model" | "gpt-3.5-turbo" | "" |

If you are interested in learning more, please check out the resources listed below.

Code Corner

Discover some fascinating code snippets or projects that showcase our work or learning journey.

bendsql - Databend Native Command Line Tool

We previously introduced bendsql, which is the Databend native client written in Go. Now it is being rewritten in Rust and is currently in the experimental stage.

The Rust version not only provides support for REST API but also supports Flight SQL.

bendsql> select avg(number) from numbers(10);

SELECT
avg(number)
FROM
numbers(10);

┌───────────────────┐
avg(number)
│ Nullable(Float64)
├───────────────────┤
4.5
└───────────────────┘

1 row in 0.259 sec. Processed 10 rows, 10B (38.59 rows/s, 308B/s)

We are excited about the progress of the Rust version and look forward to sharing more updates with you soon! Feel free to try it out and give us feedback. For more information, follow the resources listed below.

Highlights

Here are some noteworthy items recorded here, perhaps you can find something that interests you.

  • Databend leverages a RBAC model to secure your data. Read Docs | Access Control Privileges to learn about Global privileges and Object-specific privileges in Databend.
  • Added the array_aggregate function and support more array aggregate functions, including stddev_samp, stddev_pop, stddev, std, median, approx_count_distinct, kurtosis, and skewness.
  • Added the system table system.caches to describe the cache status in Databend.
  • Databend recently introduced z3.rs to solve constraints. Z3 is a theorem prover from Microsoft Research that is usually used to solve SMT problems.

What's Up Next

We're always open to cutting-edge technologies and innovative ideas. You're more than welcome to join the community and bring them to Databend.

Release proposal: Nightly v1.2

Databend v1.2 is scheduled for release on May 15th and includes the following features:

TaskStatusComments
(Query) JSON indexing#6994IN PROGRESShigh-priority
(Query) Fulltext index#3915IN PROGRESShigh-priority
(Query+Storage) Merge Into#10174PLANhigh-priority
(Query)Window function#10810IN PROGRESS
(Query)Distributed COPY#8594PLAN
(Query)Lambda function and high-order functionsPLAN
(Storage) Fuse engine re-clusteringPLANhigh-priority
(Storage) Fuse engine segment treeIN PROGRESShigh-priority
(Storage) Fuse engine orphan data cleanupIN PROGRESShigh-priority
(Storage) Compress Fuse meta data#10265IN PROGRESShigh-priority
Embedding Model #10689IN PROGRESShigh-priority
Search optimization SQL frameworkIN PROGRESS
Distributed group by with dictionaryIN PROGRESS

Please note that this is a release plan and may be subject to change. We encourage community members to participate in the discussion and provide feedback on the plan.

Issue #11073 | Release proposal: Nightly v1.2

Please let us know if you're interested in contributing to this issue, or pick up a good first issue at https://link.databend.rs/i-m-feeling-lucky to get started.

Changelog

You can check the changelog of Databend Nightly for details about our latest developments.

Full Changelog: https://github.com/datafuselabs/databend/compare/v1.0.57-nightly...v1.1.2-nightly


🎉 Contributors
21 contributors

Thanks a lot to the contributors for their excellent work.

🎈Connect With Us

Databend is a cutting-edge, open-source cloud-native warehouse built with Rust, designed to handle massive-scale analytics.

Join the Databend Community to try, get help, and contribute!