Blog Posts
Insights and articles about cloud computing, containers, and open source.
Filter by tag
Filter by tag:
Query S3 Tables with DuckDB
DuckDB has gained a new feature in preview, that allows querying of Iceberg data in AWS S3 Tables. Setting up a S3 Table There are multiple steps which need to be performed to set up a S3 Table that can be then queried with tools like DuckDB. As the ...
Querying IP addresses and CIDR ranges with DuckDB
I had a use case that eventually required performing IP address lookups in a given list of CIDR ranges, as I maintain an open source project that gathers IP address range data from public cloud providers, and also wrote an article in my blog about an...
Chat with a Duck
A while ago I published sql-workbench.com and the accompanying blog post called "Using DuckDB-WASM for in-browser Data Engineering". The SQL Workbench enables its users to analyze local or remote data directly in the browser. This lowers the bar rega...
Using DuckDB-WASM for in-browser Data Engineering
Introduction DuckDB, the in-process DBMS specialized in OLAP workloads, had a very rapid growth during the last year, both in functionality, but also popularity amongst its users, but also with developers that contribute many projects to the Open Sou...
Retrieving Lambda@Edge CloudWatch Logs
What is Lambda@Edge AWS Lambda@Edge is an extension of the traditional AWS Lambda service, but with a crucial twist – it brings serverless computing capabilities closer to the end-users. In essence, Lambda@Edge empowers developers to run custom code ...
List of free AWS Knowledge Badges
As the Skillbuilder website is sometimes a bit hard to navigate, here's the full list of free badges you can do on AWS Skillbuilder: AWS Knowledge: Cloud Essentials AWS Knowledge: Architecting AWS Knowledge: Serverless AWS Knowledge: Object Stora...
Serverless Maps for fun and profit
Introduction In today's data-driven world, interactive and visually appealing web-based maps have become an integral part of countless applications and services. Whether it's for navigation, location-based services, or data visualization, delivering ...
Gathering and analyzing public cloud provider IP address data with DuckDB & Observerable
As organizations increasingly adopt the public cloud, managing the networking and security aspects of cloud computing becomes more complex. One of the challenges that cloud administrators face is, especially in a hybrid cloud environment, keeping tra...
Casual data engineering, or: A poor man's Data Lake in the cloud - Part I
In the age of big data, organizations of all sizes are collecting vast amounts of information about their operations, customers, and markets. To make sense of this data, many are turning to data lakes - centralized repositories that store and manage ...
Using DuckDB to repartition parquet data in S3
Since release v0.7.1, DuckDB has the ability to repartition data stored in S3 as parquet files by a simple SQL query, which enables some interesting use cases. Why not use existing AWS services? If your data lake lives in AWS, a natural choice for ET...