<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="4.1.0">Jekyll</generator><link href="/feed.xml" rel="self" type="application/atom+xml" /><link href="/" rel="alternate" type="text/html" /><updated>2025-07-26T06:15:41-05:00</updated><id>/feed.xml</id><entry><title type="html">Why I’m stepping down as Head of Engineering</title><link href="/blog/why-im-stepping-down-as-head-of-engineering-20250726.html" rel="alternate" type="text/html" title="Why I’m stepping down as Head of Engineering" /><published>2025-07-26T05:00:00-05:00</published><updated>2025-07-26T05:00:00-05:00</updated><id>/blog/why-im-stepping-down-as-head-of-engineering</id><content type="html" xml:base="/blog/why-im-stepping-down-as-head-of-engineering-20250726.html">&lt;p&gt;Four years after founding the company, I’m stepping away from executive management at LocalStack.
It was not an easy decision, but came after months of reflecting, and the honest realization that I was drifting away from the kind of impact that gives me energy.
I’m now moving back into a cross-team IC-focused role, and it’s a good opportunity to tell this story, in the hopes it might help others in similar situations find the courage to make a change!&lt;/p&gt;

&lt;h1 id=&quot;the-road-to-now&quot;&gt;The road to now&lt;/h1&gt;

&lt;h2 id=&quot;bootstrapping-localstack&quot;&gt;Bootstrapping LocalStack&lt;/h2&gt;

&lt;p&gt;It’s April 2021, and we’re a couple of months away from incorporating.
I reach out to some of the best people I’ve worked with in the past, and convince them to quit their well-paying and secure job and instead join a scrappy little startup with a small seed funding of $3M.
To my initial surprise, almost everyone I reach out to is all-in.
The energy this gives me is hard to explain, but it resolves all the doubts I had about letting my other ventures go to focus entirely on LocalStack.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;/assets/img/blog/2025-07-26-why-im-stepping-down-as-head-of-engineering/localstack-founders-at-reinvent-2022.jpeg&quot; data-toggle=&quot;lightbox&quot; data-gallery=&quot;blog-gallery&quot;&gt;&lt;img src=&quot;/assets/img/blog/2025-07-26-why-im-stepping-down-as-head-of-engineering/localstack-founders-at-reinvent-2022.jpeg&quot; alt=&quot;A photo of the LocalStack founders visiting re:Invent in 2022&quot; class=&quot;img-blog&quot; /&gt;&lt;/a&gt;
&lt;em&gt;LocalStack Founders at re:Invent in 2022 (fltr: Thomas, Gerta, Waldemar)&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Together with this rag-tag group of six engineers and my two co-founders, I roll up my sleeves, and set out to do the prep work for our long growth journey ahead, transforming the LocalStack OSS project, Waldemar’s incredible creation, into a world-class product platform.&lt;/p&gt;

&lt;p&gt;Almost all of those folks are still in the company by the way.
Having grown to senior, staff, team lead, and engineering manager roles, they continue to be the bedrock of our engineering team, and I’m incredibly grateful to them.&lt;/p&gt;

&lt;h2 id=&quot;the-joy-and-cost-of-generalist-leadership&quot;&gt;The joy and cost of generalist leadership&lt;/h2&gt;

&lt;h3 id=&quot;being-a-lead-developer&quot;&gt;Being a Lead Developer&lt;/h3&gt;

&lt;p&gt;I spend a lot of my time in the code base, finding bottlenecks for our growing team.
While building new product features myself, I also start solving some higher-order problems.
Like building a framework for &lt;a href=&quot;https://www.youtube.com/watch?v=WSD20OIPRUA&quot;&gt;generalizing the development of AWS service emulators&lt;/a&gt;, which has since transformed how we work at LocalStack.
I also spearhead initiatives like dogfooding LocalStack, where we deploy and test our AWS-backed platform on LocalStack.
This also marks the beginning of the &lt;a href=&quot;https://github.com/localstack/localstack-extensions/&quot;&gt;LocalStack extension system&lt;/a&gt;, which allows us to emulate our payment backend Stripe directly in LocalStack.
It is innovative and deeply technical work, and with our small team of highly skilled and driven engineers, the progress we make feels spectacular.&lt;/p&gt;

&lt;h3 id=&quot;being-a-company-architect&quot;&gt;Being a Company Architect&lt;/h3&gt;

&lt;p&gt;With my co-founders focusing heavily on the business, in particular customer and investor relations, my natural focus is facing inwards, essentially greenfielding the company with this young and highly driven team.
I can focus working on the things that drew me to found a company in the first place: to create an interesting place to work, where people &lt;em&gt;show up&lt;/em&gt; because they can work on things they care about, with people they enjoy working with.
So naturally I spend a lot of time thinking about &lt;a href=&quot;https://localstack.notion.site/candidate-handbook-x-localstack#192fc2a234318052816ac6abcd5609b3&quot;&gt;culture&lt;/a&gt; and designing ways of remote work collaboration.&lt;/p&gt;

&lt;p&gt;We lack executive-level or director-level people with deep industry experience, and being first time founders, most of what we do and how we do things comes from first principles and our intuitions.
So I immerse myself in literature and learn how other companies do things.
I learn about &lt;a href=&quot;https://www.fastcompany.com/1663137/how-3m-gave-everyone-days-off-and-created-an-innovation-dynamo&quot;&gt;3M’s 15% time&lt;/a&gt;, &lt;a href=&quot;https://www.valvesoftware.com/company/Valve_Handbook_LowRes.pdf&quot;&gt;Valve’s flat org structure&lt;/a&gt;, &lt;a href=&quot;https://basecamp.com/shapeup&quot;&gt;Basecamp’s Shape Up&lt;/a&gt;, &lt;a href=&quot;https://careersatdoordash.com/blog/wedash-doordash-employee-program-how-does-it-work/&quot;&gt;DoorDash’s “We Dash”&lt;/a&gt;, &lt;a href=&quot;https://github.com/kubernetes/community/blob/master/governance.md&quot;&gt;self-organizing projects like Kubernetes&lt;/a&gt;, and many other ideas, which inspire processes we design with the team: our Incubator program to foster bottom-up innovation, regular hackathons to try out crazy new things on company time, Mentors instead of Managers, Special Interest Groups, and a “We Support” support rotation where everyone (co-founders included) is put into the shoes of the customer, all of which shape the culture at LocalStack, and creates a place where people love to come to work.&lt;/p&gt;

&lt;p&gt;All these ideas, cherry-picked and adapted from other successful and unique ventures, end up forming “The LocalStack Way”.
Here’s a collage from that presentation that I should probably write a separate post about:&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;/assets/img/blog/2025-07-26-why-im-stepping-down-as-head-of-engineering/the-localstack-way.jpg&quot; data-toggle=&quot;lightbox&quot; data-gallery=&quot;blog-gallery&quot;&gt;&lt;img src=&quot;/assets/img/blog/2025-07-26-why-im-stepping-down-as-head-of-engineering/the-localstack-way.jpg&quot; alt=&quot;The LocalStack Way presentation&quot; class=&quot;img-blog&quot; /&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3 id=&quot;everything-everywhere-all-at-once&quot;&gt;Everything everywhere all at once&lt;/h3&gt;

&lt;p&gt;As founders we tend to be everywhere at once, shaping systems and teams across the company.
In this undefined role, I execute quickly with the team, lay out the technical vision for the core product, make process and hiring decisions across functions, and start building out other teams.&lt;/p&gt;

&lt;p&gt;While my co-founders are building out other areas of the organization like HR, sales, and finance, I step in where there are gaps.
We hadn’t set up a clear reporting structure at that point, but effectively I’m managing Engineering, DevRel and Support, scaling the teams, and experiment and build together with them.
We don’t have a product team yet, so a lot of product work is driven by myself and the CTO.&lt;/p&gt;

&lt;p&gt;Cycling between Engineering, Support, DevRel, Product, Leadership and general Founder activity takes a lot of mental energy.
But being so plugged into everything makes designing transversal systems and being the glue between teams much easier.
I constantly have the full context allowing me to give clear and coherent directions.
Overall things are moving fast, they’re challenging in a good way, and people are genuinely excited to be here.
It’s very gratifying.&lt;/p&gt;

&lt;h3 id=&quot;becoming-specialized&quot;&gt;Becoming specialized&lt;/h3&gt;

&lt;p&gt;With all our growing teams, it’s becoming more evident that we soon need people who have actual experience in scaling these org functions.
We’re around 35 people at this point, and we start hiring a Head of Sales and a Director of Engineering.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;/assets/img/blog/2025-07-26-why-im-stepping-down-as-head-of-engineering/img-20240117.jpg&quot; data-toggle=&quot;lightbox&quot; data-gallery=&quot;blog-gallery&quot;&gt;&lt;img src=&quot;/assets/img/blog/2025-07-26-why-im-stepping-down-as-head-of-engineering/img-20240117.jpg&quot; alt=&quot;Our second team retreat in 2024&quot; class=&quot;img-blog&quot; /&gt;&lt;/a&gt;
&lt;em&gt;LocalStack Team in January 2024&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;As the teams grow and the foundations are set, I find myself drifting away from the undefined “company architect” role I loved and thrived in.
My scope progressively narrows to a Head of Engineering role, responsible primarily for scaling delivery, managing people, and keeping the machine running.
I’m no longer interviewing every person who joins the company, but focus now primarily on engineering.
More and more of my work happens through others, managing managers and team leads.&lt;/p&gt;

&lt;p&gt;I’m also spending more time on foundational things in the engineering department, like designing a career and performance evaluation framework, or setting up OKRs for the next quarter.
I like working on those things, but I notice that the work no longer translates naturally to other parts of the organization as it has previously when my role was less specialized.&lt;/p&gt;

&lt;h2 id=&quot;breaking-points&quot;&gt;Breaking points&lt;/h2&gt;

&lt;h3 id=&quot;drifting-from-my-passion&quot;&gt;Drifting from my passion&lt;/h3&gt;

&lt;p&gt;In 2024, I spend most of my August, September, and October either in 1:1s, candidate interviews, or planning and decision making meetings.
I actually hit a low point in technical contributions, which I know I shouldn’t be doing in my role in the first place.
Clearly this is not a problem with the role per-se, but with me being terrible at saying no to things.
I care too much about everything, and my founder instincts keep pulling me into being everywhere at once, although it’s now completely incompatible with the reality of our, at this point, 70 person company.&lt;/p&gt;

&lt;p&gt;With my calendar now being basically fully booked two weeks in advance, it’s becoming harder and harder to carve out what I now know as “&lt;a href=&quot;https://paulgraham.com/makersschedule.html&quot;&gt;maker time&lt;/a&gt;”.
Whenever I do, I feel guilty for neglecting things I should be focusing on more.
And whenever I lean fully into management, despite doing it well, I feel restless and somewhat disconnected from the work that had brought me here in the first place.
Charity Majors calls this &lt;a href=&quot;https://charity.wtf/2017/05/11/the-engineer-manager-pendulum/&quot;&gt;the engineer/manager pendulum&lt;/a&gt;, and some thrive in the swing between modes.&lt;/p&gt;

&lt;p&gt;Having spoken to other founders and technical leaders, it’s what many of them end up learning: being good at management doesn’t mean it’s where you’ll do your best or happiest work.
I realize that in becoming more specialized, I was also becoming less impactful in the way that matters to me the most.&lt;/p&gt;

&lt;h3 id=&quot;losing-sight&quot;&gt;Losing sight&lt;/h3&gt;

&lt;p&gt;Together with our expanded leadership team, we organize a management offsite.
Two full days spent discussing our many challenges and growth pains, mapping out the next year, and preparing for the changes that Series-A growth will demand.&lt;/p&gt;

&lt;p&gt;In earlier phases, moments like this absolutely filled me with energy.
I could see with clarity what was needed, and I could see myself stepping into a role that was required of me with purpose and deliberation.
But this time is different.
Something had shifted: I realize I no longer see myself in the role that this next chapter requires, which had become one of managing and operating at a distance from the work I loved: shaping systems, teams, and products hands-on, all while providing vision and guidance to the teams.
But with the way the leadership team is developing, and my two co-founders holding three executive roles: CEO, CTO, and COO, I realize that I’m stuck and my energy and values are pulling me in a different direction.&lt;/p&gt;

&lt;h3 id=&quot;being-in-the-trenches-again&quot;&gt;Being in the trenches again&lt;/h3&gt;

&lt;p&gt;It’s late October, and I’m called into our Vienna office to deal with yet another office malfunction: water dripping through the ceiling.
Again.
But it allows me to spend time with everyone like in the early days.
I make a round through the office, ask everyone how they’re doing.
Some conversations take a minute, others turn into spontaneous paring sessions.
I work through an issue with them, and they end up with more clarity and determination than before.
I leave the room full of quiet satisfaction, not because of status or authority, but because I was &lt;em&gt;in it&lt;/em&gt; again.
Present, useful, and technical.
Something I realize I’ve been missing deeply over the past months.
I didn’t even notice the water dripping onto my laptop.&lt;/p&gt;

&lt;p&gt;The next day, I start writing down why I’m going to step away from executive management, and start preparing for the meeting with my co-founders to let them know.&lt;/p&gt;

&lt;h1 id=&quot;re-defining-impact-for-myself&quot;&gt;Re-defining impact for myself&lt;/h1&gt;

&lt;h2 id=&quot;disconnect-to-reconnect&quot;&gt;Disconnect to reconnect&lt;/h2&gt;

&lt;p&gt;After months of planning and handing over all of my responsibilities, I took a two-month leave.
It felt strange at first, and of course I still opened my laptop regularly, given my irrational fear of the thousands of Slack notifications that I would miss out on if I didn’t.
This was before I learned about Shift-Esc and had developed the emotional capacity to actually use it.
And because I lack any and all self-control, the only way I could finally disconnect was by logging out of Slack on all my devices.&lt;/p&gt;

&lt;p&gt;It was the first time in years I wasn’t surrounded by constant input, decision-making pressure, and fire alarms.
I distracted myself with other technical projects, and in the absence of daily notifications, a quietness started taking over that allowed me to detach and reflect more deeply and honestly about the decision I had made.&lt;/p&gt;

&lt;h2 id=&quot;its-not-actually-stepping-down&quot;&gt;It’s not actually stepping down&lt;/h2&gt;

&lt;p&gt;Stepping away from a position of status and authority is incredibly difficult.
We’ve been conditioned to equate executive titles with success, so it’s no surprise when doubt starts to creep in that undermines your confidence.
Did I fail at my job? Am I giving up and letting people down? Am I damaging my career?&lt;/p&gt;

&lt;p&gt;But there’s a growing counter-narrative in the tech world.
There is the well-known example of &lt;a href=&quot;https://www.hashicorp.com/en/blog/mitchell-s-new-role-at-hashicorp&quot;&gt;Mitchell Hashimoto’s decision to step down as CTO of HashiCorp&lt;/a&gt; to become a Senior IC.
There are others too.
Former Artsy CTO &lt;a href=&quot;https://code.dblock.org/2019/11/17/the-pros-and-cons-of-going-from-management-back-to-ic.html&quot;&gt;Daniel Doubrovkine&lt;/a&gt;, now Principal Engineer at AWS, &lt;a href=&quot;https://stackoverflow.blog/2021/12/29/moving-from-designer-to-developer-stack-overflow/&quot;&gt;Kristina Lustig&lt;/a&gt;, former Director of Design at StackOverflow, or &lt;a href=&quot;https://pradiptabanerjee.medium.com/transitioning-back-to-an-individual-contributor-role-795e783d29f8&quot;&gt;Pradipta Banerjee&lt;/a&gt;, former CTO now Principal Engineer at Red Hat.
They all shared their journey from management back to hands-on engineering.
These stories seem like outliers, drowned out by the noise of LinkedIn posts celebrating promotions, but they all share an underlying theme.
Each of them had reached a point where traditional definitions of success no longer aligned with their sense of fulfillment.
They had confronted what psychologists call the &lt;a href=&quot;https://www.exceptionalfutures.com/the-arrival-fallacy/&quot;&gt;arrival fallacy&lt;/a&gt;: the belief that attaining a particular goal will lead to long-term happiness.&lt;/p&gt;

&lt;p&gt;Having now recognized this as a trap, I was able to re-frame my own decision.
Instead of asking “What am I stepping away from”, I asked myself “what gives me energy, and what takes it away?”, which was much more productive.
Was I really at my most impactful in back-to-back meetings, managing OKRs and team performance? Or was I just keeping the machine running, at the cost of the creative work that played to my strengths and brought me here in the first place?&lt;/p&gt;

&lt;h2 id=&quot;having-authority-vs-being-an-authority&quot;&gt;Having authority vs being an authority&lt;/h2&gt;

&lt;p&gt;I’ve had frustrating experiences in my early career where I couldn’t convince people in power to address problems that seemed so glaringly evident to me, while at the same time lacking the agency to solve them in my own way.
I learned that this is &lt;a href=&quot;https://www.youtube.com/watch?v=BooPVxoRmEY#t=14m&quot;&gt;common among people stuck in middle management&lt;/a&gt;.
Whenever I finally did get a position of authority, I was able to drive change quickly and get great results that way.
This entrenched my belief that holding a position of authority was the key to having high impact.
It’s clear to me now that this was another reason I wanted to found a company in the first place.&lt;/p&gt;

&lt;p&gt;Over time, though, I realized that having authority and being an authority aren’t the same thing.
Titles can create compliance, but not necessarily conviction.
Yes, when people are contractually obliged to follow your lead, change happens fast, but I’ve found that sustained influence doesn’t come from org charts.
It comes from vision, trust, and building systems people actually want to work in.
Of course these aren’t mutually exclusive, but most of us aren’t really chasing &lt;em&gt;authority,&lt;/em&gt; we’re chasing &lt;em&gt;impact&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;It comes down to a few simple habits.
Be around when it matters, listen to people’s concerns and be their advocate, support and elevate people around you by providing input that leads to better outcomes for them, and inspire by leading by example.
To me, this means being hands-on and regularly working shoulder to shoulder.
Or, as one of my colleagues succinctly put it, “just sharing your wisdom and then fucking off”.
Doing those things will make you an agent of change, regardless of title.&lt;/p&gt;

&lt;h2 id=&quot;making-room-for-others&quot;&gt;Making room for others&lt;/h2&gt;

&lt;p&gt;Founders often struggle to let go because they conflate personal value with holding on to certain responsibilities.
But roles evolve, and the person who’s best for the job at one stage of a company’s growth may not be the best fit for the next.
All this wasn’t just about me.&lt;/p&gt;

&lt;p&gt;We had built a strong foundation of systems, culture, and technical vision that no longer depended on me to function.
At our size and pace, we needed more leaders whose strengths lay in scaling delivery, applying repeatable management playbooks, navigating teams through organizational change, and creating endless KPI reports while keeping their sanity intact.
These are things I was very capable at, but not things I thrived in, and clinging to the seat would have only limited the team.
Stepping away created space for new people and new perspectives, while allowing me to focus on cross-team technical leadership, where I knew I could bring the most value to the team and the company.&lt;/p&gt;

&lt;p&gt;I’ve always believed in embracing change, whether in our product, processes, or leadership.
And I’ve always sought out opportunities to experiment, to learn from others, and be inspired by the experience they bring.
Our engineering leadership team is now more distributed, with multiple Directors and Engineering Managers, and they strengthen the company in ways I couldn’t have achieved alone.&lt;/p&gt;

&lt;h1 id=&quot;learnings&quot;&gt;Learnings&lt;/h1&gt;

&lt;p&gt;If you’re considering a change like the one I made, there are a number of learnings that I can offer as advice.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Be honest with yourself:&lt;/strong&gt; Why do you want a particular role in the first place? Is it impact? Perhaps the work you’ll get to do? Is it about advancing your career? Or is it the allure of Power? Money? Status? I’ve stopped judging people for wanting these things, but be honest with yourself about your motivations, and ask yourself whether they are leading you to meaningful impact and lasting happiness.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;You are more than your job:&lt;/strong&gt; It should be obvious: but you should not attach your self-worth to a role or a job title.
It’s very hard to relinquish a title and not feel inadequate in a culture that glorifies titles and promotions.
You can read more about how this culture has affected an entire generation in &lt;a href=&quot;https://waitbutwhy.com/2013/09/why-generation-y-yuppies-are-unhappy.html&quot;&gt;Tim Urban’s blog post on “Why Generation Y Yuppies Are Unhappy”&lt;/a&gt;.
Real self-worth isn’t found on a career ladder; it’s in the value you create, the people you elevate, and the integrity of your actions.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Build a support network&lt;/strong&gt;: A curse of executive leadership is that it can become lonely, as you have few peers at the company you can open up to and who understand what you’re going through.
Build a professional support network outside the company.
Find peers who you feel safe to be vulnerable around, and to talk honestly and openly about what you’re going through.
You’ll be surprised how many people are going through similar things, and how they open up when you show a bit of vulnerability yourself.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Acknowledge the challenges but focus on the opportunities:&lt;/strong&gt; Suddenly not being part of the “inner circle” is difficult.
There’s a sense that you’re losing control and are no longer informed, but at the same time, most of the things that demanded your attention were probably the things that drove you to change your situation in the first place.
As Daniel Doubrovkine put it: “&lt;em&gt;I [knew] all the secrets […] and knew when someone important was quitting or being fired.
This is all gone, but so are the agonizing headaches, such as not being able to tell someone that they are about to get laid off&lt;/em&gt;”.
So you’re not missing out, you’re reclaiming your time for things that matter to you the most.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Re-design your work day:&lt;/strong&gt; use the opportunity to redesign how you work.
For example, you probably don’t have any direct reports anymore, so you have no obligations to have regular 1:1s, but that doesn’t keep you from having them, and they can be extremely useful to keep your finger on the pulse! Make them time bound, and purpose driven.
They repeat for eight weeks and then end.
Maybe it’s to pair weekly on architecture during big initiatives, or to coach someone hands-on through a difficult period?&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;You only live once&lt;/strong&gt;: Hone in on things that matter the most, but don’t compromise on the things you love doing.
Everything ends someday.
The jobs, the titles, and once the last black hole has evaporated and the universe has succumbed to heat death, even your company.
What remains is the journey you lived, the legacy you left, and the impact you had on the people around you.&lt;/p&gt;</content><author><name></name></author><category term="Career" /><summary type="html">Four years after founding the company, I’m stepping away from executive management at LocalStack. It was not an easy decision, but came after months of reflecting, and the honest realization that I was drifting away from the kind of impact that gives me energy. I’m now moving back into a cross-team IC-focused role, and it’s a good opportunity to tell this story, in the hopes it might help others in similar situations find the courage to make a change!</summary></entry><entry><title type="html">Syncing Notion databases into Tinybird using notion-objects</title><link href="/blog/syncing-notion-database-into-tinybird-with-notion-objects-20221017.html" rel="alternate" type="text/html" title="Syncing Notion databases into Tinybird using notion-objects" /><published>2022-10-17T03:00:00-05:00</published><updated>2022-10-17T03:00:00-05:00</updated><id>/blog/syncing-notion-database-into-tinybird-with-notion-objects</id><content type="html" xml:base="/blog/syncing-notion-database-into-tinybird-with-notion-objects-20221017.html">&lt;p&gt;In this blog post, I want to show you how you can, with just a few lines of Python code using the &lt;a href=&quot;https://github.com/thrau/notion-objects&quot;&gt;notion-objects&lt;/a&gt; library and the Tinybird API,
synchronize data from Notion databases into Tinybird data sources.&lt;/p&gt;

&lt;p&gt;I’ve become a big fan of both &lt;a href=&quot;https://notion.so&quot;&gt;Notion&lt;/a&gt; and &lt;a href=&quot;https://tinybird.co&quot;&gt;Tinybird&lt;/a&gt;.
They are excellent products and I use them daily at &lt;a href=&quot;https://localstack.cloud&quot;&gt;LocalStack&lt;/a&gt;.
We’ve been &lt;a href=&quot;https://www.tinybird.co/case-studies/localstack-relies-on-tinybird-to-understand-and-improve-their-product&quot;&gt;using Tinybird&lt;/a&gt; as a consolidated data warehouse for a while, and query it for all sorts of analytics.
We’ve also been steadily building up data in various Notion databases, including our project roadmap or customer incident tracking, that we wanted to start running queries against.
So it was clear we had to start syncing data from Notion into Tinybird.
I spent some time last weekend to create a library to make it easier to get data out of Notion.
We now use the library in scheduled AWS Lambda functions that do the synchronization.
After reading this article, you should be able to build something similar.&lt;/p&gt;

&lt;h2 id=&quot;prerequisites&quot;&gt;Prerequisites&lt;/h2&gt;

&lt;p&gt;Requirements for this:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Python 3&lt;/li&gt;
  &lt;li&gt;A &lt;a href=&quot;https://www.notion.so/signup&quot;&gt;Notion account&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;A &lt;a href=&quot;https://ui.tinybird.co/signup&quot;&gt;Tinybird account&lt;/a&gt; and workspace&lt;/li&gt;
  &lt;li&gt;A basic understanding of the three&lt;/li&gt;
&lt;/ul&gt;

&lt;h3 id=&quot;creating-a-notion-integration&quot;&gt;Creating a Notion integration&lt;/h3&gt;

&lt;p&gt;I’m assuming you have a notion account and have some familiarity with it.
To create an API token for Notion, we first need to create what Notion calls an integration,
and connect the integration to the database we want to synchronize.
The &lt;a href=&quot;https://developers.notion.com/docs/getting-started&quot;&gt;Getting Started guide&lt;/a&gt; in the Notion developer docs explains this very concisely.&lt;/p&gt;

&lt;p&gt;Here’s my notion database I use for demonstration.
I also highlighted where to find the database id (needed later).&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;/assets/img/blog/2022-10-17-syncing-notion-database-into-tinybird-with-notion-objects/notion-task-list.png&quot; data-toggle=&quot;lightbox&quot; data-gallery=&quot;blog-gallery&quot;&gt;&lt;img src=&quot;/assets/img/blog/2022-10-17-syncing-notion-database-into-tinybird-with-notion-objects/notion-task-list.png&quot; alt=&quot;Notion database - a task list&quot; class=&quot;img-blog&quot; /&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2 id=&quot;notion-objects&quot;&gt;notion-objects&lt;/h2&gt;

&lt;p&gt;&lt;a href=&quot;https://github.com/thrau/notion-objects&quot;&gt;notion-objects&lt;/a&gt; is a data mapper for mapping Notion database records to Python objects.
To connect to Notion, it uses &lt;a href=&quot;https://github.com/ramnes/notion-sdk-py&quot;&gt;notion-sdk-py&lt;/a&gt;, a Notion SDK for Python.&lt;/p&gt;

&lt;p&gt;Simply install notion-objects using pip:&lt;/p&gt;

&lt;div class=&quot;language-sh highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;pip &lt;span class=&quot;nb&quot;&gt;install &lt;/span&gt;notion-objects
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;With &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;notion-objects&lt;/code&gt;, you can easily iterate over all records in a database,
and access Notion database fields like Python object attributes.
You can build your own data models and mappings, which is explained in the &lt;a href=&quot;https://github.com/thrau/notion-objects/blob/main/README.md&quot;&gt;notion-objects README&lt;/a&gt;, but the default dynamic mapping works pretty well for basic use cases.&lt;/p&gt;

&lt;p&gt;Here’s some example code.&lt;/p&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;kn&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;nn&quot;&gt;os&lt;/span&gt;
&lt;span class=&quot;kn&quot;&gt;from&lt;/span&gt; &lt;span class=&quot;nn&quot;&gt;notion_client&lt;/span&gt; &lt;span class=&quot;kn&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Client&lt;/span&gt;
&lt;span class=&quot;kn&quot;&gt;from&lt;/span&gt; &lt;span class=&quot;nn&quot;&gt;notion_objects&lt;/span&gt; &lt;span class=&quot;kn&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Database&lt;/span&gt;

&lt;span class=&quot;k&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;main&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;():&lt;/span&gt;
	&lt;span class=&quot;n&quot;&gt;notion_token&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;&quot;&lt;/span&gt; &lt;span class=&quot;c1&quot;&gt;# notion integration token goes here
&lt;/span&gt;	&lt;span class=&quot;n&quot;&gt;database_id&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;&quot;&lt;/span&gt; &lt;span class=&quot;c1&quot;&gt;# add your database ID here
&lt;/span&gt;
    &lt;span class=&quot;c1&quot;&gt;# create the notion client
&lt;/span&gt;    &lt;span class=&quot;n&quot;&gt;client&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Client&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;auth&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;notion_token&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;c1&quot;&gt;# the notion-objects database
&lt;/span&gt;    &lt;span class=&quot;n&quot;&gt;db&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Database&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;database_id&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;client&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;k&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;record&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;db&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
        &lt;span class=&quot;k&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;record&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;id&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;record&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Name&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;record&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Assignee&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;record&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Status&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

&lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;__name__&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;==&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;'__main__'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;main&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;For my database, this prints:&lt;/p&gt;
&lt;div class=&quot;language-console highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;go&quot;&gt;10289cdc-7987-4f2d-8f6d-be8ab7d4308e Create notion workspace ['Thomas R.'] Done 🙌
44d54963-c2ef-40de-80ef-d25fe6df444f Publish blog post [] To Do
8b8dd97e-2e7f-470a-b076-3958100293f6 Write blog post about notion-objects ['Thomas R.'] Doing
964da614-f41f-4930-aed6-ca50f10603ee Take Fig on a walk [] Doing
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Record objects also have &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;to_json&lt;/code&gt; and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;to_dict&lt;/code&gt; converters, that will later come in handy.&lt;/p&gt;

&lt;h2 id=&quot;getting-data-into-tinybird&quot;&gt;Getting data into Tinybird&lt;/h2&gt;

&lt;p&gt;The events endpoint of Tinybird is convenient because it accepts NDJSON and will also create the datasource if it does not exist.
So all we need is a datasource name, an NDJSON payload, and a POST request to the events endpoint.&lt;/p&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;    &lt;span class=&quot;c1&quot;&gt;# build an ndjson batch that we later sync into tinybird
&lt;/span&gt;    &lt;span class=&quot;n&quot;&gt;batch&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;[]&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;record&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;db&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
        &lt;span class=&quot;c1&quot;&gt;# flat=True flattens date range properties into two columns
&lt;/span&gt;        &lt;span class=&quot;n&quot;&gt;batch&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;append&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;record&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;to_json&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;flat&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;True&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;))&lt;/span&gt;

    &lt;span class=&quot;n&quot;&gt;data&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;se&quot;&gt;\n&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;join&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;batch&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;data&lt;/code&gt; will now contain a new-line delimited JSON (NDJSON) payload, which we can post directly to the events endpoint:&lt;/p&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;    &lt;span class=&quot;c1&quot;&gt;# converts something like &quot;My *fantastic* database&quot; into &quot;my__fantastic__database&quot;, which is a vald tinybird datasource name
&lt;/span&gt;    &lt;span class=&quot;n&quot;&gt;tinybird_datasource&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;re&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;sub&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;'[^0-9a-zA-Z]+'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;'_'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;db&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;title&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;strip&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()).&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;lower&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt;

    &lt;span class=&quot;c1&quot;&gt;# post the batch into tinybird and create the table if it doesn't exist
&lt;/span&gt;    &lt;span class=&quot;n&quot;&gt;requests&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;post&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;
        &lt;span class=&quot;s&quot;&gt;'https://api.tinybird.co/v0/events'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;params&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
            &lt;span class=&quot;s&quot;&gt;'name'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tinybird_datasource&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
            &lt;span class=&quot;s&quot;&gt;'token'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tinybird_token&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;p&quot;&gt;},&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;data&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;data&lt;/span&gt;
    &lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h3 id=&quot;code-so-far&quot;&gt;Code so far&lt;/h3&gt;

&lt;p&gt;Let’s put together what we have so far:&lt;/p&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;kn&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;nn&quot;&gt;re&lt;/span&gt;

&lt;span class=&quot;kn&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;nn&quot;&gt;requests&lt;/span&gt;
&lt;span class=&quot;kn&quot;&gt;from&lt;/span&gt; &lt;span class=&quot;nn&quot;&gt;notion_client&lt;/span&gt; &lt;span class=&quot;kn&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Client&lt;/span&gt;

&lt;span class=&quot;kn&quot;&gt;from&lt;/span&gt; &lt;span class=&quot;nn&quot;&gt;notion_objects&lt;/span&gt; &lt;span class=&quot;kn&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Database&lt;/span&gt;


&lt;span class=&quot;k&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;main&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;():&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;notion_token&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;secret_&quot;&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;tinybird_token&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;p.&quot;&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;database_id&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;c59fec7c9a4b43b0b069b0c0aa843054&quot;&lt;/span&gt;

    &lt;span class=&quot;c1&quot;&gt;# create the notion client
&lt;/span&gt;    &lt;span class=&quot;n&quot;&gt;client&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Client&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;auth&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;notion_token&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;c1&quot;&gt;# the notion-objects database
&lt;/span&gt;    &lt;span class=&quot;n&quot;&gt;db&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Database&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;database_id&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;client&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;c1&quot;&gt;# build an ndjson batch that we later sync into tinybird
&lt;/span&gt;    &lt;span class=&quot;n&quot;&gt;batch&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;[]&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;record&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;db&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;batch&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;append&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;record&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;to_json&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;flat&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;True&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;))&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;data&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;se&quot;&gt;\n&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;join&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;batch&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;c1&quot;&gt;# convert the title of the notion database into a valid tinybird datasource name
&lt;/span&gt;    &lt;span class=&quot;n&quot;&gt;tinybird_datasource&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;re&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;sub&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;'[^0-9a-zA-Z]+'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;'_'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;db&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;title&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;strip&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()).&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;lower&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt;

    &lt;span class=&quot;c1&quot;&gt;# post the batch into tinybird and create the table if it doesn't exist
&lt;/span&gt;    &lt;span class=&quot;n&quot;&gt;requests&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;post&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;
        &lt;span class=&quot;s&quot;&gt;'https://api.tinybird.co/v0/events'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;params&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
            &lt;span class=&quot;s&quot;&gt;'name'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tinybird_datasource&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
            &lt;span class=&quot;s&quot;&gt;'token'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tinybird_token&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;p&quot;&gt;},&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;data&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;data&lt;/span&gt;
    &lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;


&lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;__name__&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;==&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;'__main__'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;main&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;If you run this script, this is what you get in your Tinybird workspace:&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;/assets/img/blog/2022-10-17-syncing-notion-database-into-tinybird-with-notion-objects/tinybird-initial-import.png&quot; data-toggle=&quot;lightbox&quot; data-gallery=&quot;blog-gallery&quot;&gt;&lt;img src=&quot;/assets/img/blog/2022-10-17-syncing-notion-database-into-tinybird-with-notion-objects/tinybird-initial-import.png&quot; alt=&quot;Notion data in tinybird&quot; class=&quot;img-blog&quot; /&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2 id=&quot;syncing-changes&quot;&gt;Syncing changes&lt;/h2&gt;

&lt;p&gt;Next we want to deal with syncing data continuously.
Since Notion currently does not provide webhooks to react to changes, we need to run a scheduled synchronization job.
What would this job do?
One way would be to truncate the datasource, and then run the full insert every time.&lt;/p&gt;

&lt;p&gt;Alternatively, if we want to keep track of changes, we could only insert those notion objects that have changed since the last update.&lt;/p&gt;

&lt;h3 id=&quot;get-the-time-of-the-latest-synced-change&quot;&gt;Get the time of the latest synced change&lt;/h3&gt;

&lt;p&gt;notion-objects creates a default column &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;last_edited_time&lt;/code&gt; that maps to the notion page’s last edit timestamp.
We could first query this from the the table using the &lt;a href=&quot;https://www.tinybird.co/docs/api-reference/query-api.html#get--v0-sql&quot;&gt;Tinybird SQL endpoint &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;GET /v0/sql&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;    &lt;span class=&quot;n&quot;&gt;r&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;requests&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;get&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;
        &lt;span class=&quot;s&quot;&gt;f'https://api.tinybird.co/v0/sql'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;params&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
            &lt;span class=&quot;s&quot;&gt;'q'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;f&quot;SELECT max(last_edited_time) FROM &lt;/span&gt;&lt;span class=&quot;si&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;tinybird_datasource&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
            &lt;span class=&quot;s&quot;&gt;'token'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tinybird_token&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
    &lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;r&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;text&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;  &lt;span class=&quot;c1&quot;&gt;# 2022-10-16 19:36:00
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;An alternative could be to use the datasource’s operations log, but this is out of scope of this blog post.&lt;/p&gt;

&lt;h3 id=&quot;build-a-dynamic-notion-query-filter&quot;&gt;Build a dynamic notion query filter&lt;/h3&gt;

&lt;p&gt;Now we want to fetch only those notion objects from the database that have been edited since the last time the data was synced.
To that end, we dynamically build a &lt;a href=&quot;https://developers.notion.com/reference/post-database-query-filter&quot;&gt;notion query filter&lt;/a&gt;, based on the input of the previous query.
If the database doesn’t exist yet, the Tinybird SQL query may fail, so we guard against that case.&lt;/p&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;    &lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;not&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;r&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;ok&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;or&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;not&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;r&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;text&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;strip&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;():&lt;/span&gt;
        &lt;span class=&quot;c1&quot;&gt;# probably the table was not found, so we just use None
&lt;/span&gt;        &lt;span class=&quot;n&quot;&gt;query&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{}&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;else&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
        &lt;span class=&quot;c1&quot;&gt;# build a query that fetches only those pages that were edited
&lt;/span&gt;        &lt;span class=&quot;c1&quot;&gt;# since the last time the database was synced
&lt;/span&gt;        &lt;span class=&quot;n&quot;&gt;query&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
            &lt;span class=&quot;s&quot;&gt;&quot;filter&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
                &lt;span class=&quot;s&quot;&gt;&quot;timestamp&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;last_edited_time&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
                &lt;span class=&quot;s&quot;&gt;&quot;last_edited_time&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;after&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;r&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;text&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;strip&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()},&lt;/span&gt;
            &lt;span class=&quot;p&quot;&gt;},&lt;/span&gt;
        &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now you can use notion-objects &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;Database.query&lt;/code&gt; method to iterate over the result:&lt;/p&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;    &lt;span class=&quot;n&quot;&gt;batch&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;[]&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;record&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;db&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;query&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;query&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;):&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;batch&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;append&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;record&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;to_json&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;flat&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;True&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;))&lt;/span&gt;

    &lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;not&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;batch&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
        &lt;span class=&quot;k&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;batch is empty, probably no data was edited&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
        &lt;span class=&quot;k&quot;&gt;return&lt;/span&gt;

    &lt;span class=&quot;n&quot;&gt;data&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;se&quot;&gt;\n&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;join&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;batch&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h3 id=&quot;de-duplicating-data&quot;&gt;De-duplicating data&lt;/h3&gt;

&lt;p&gt;Obviously we now have records with the same ID multiple times.
Once for every sync if it has changed since the last sync.
There are several ways you can deal with duplicate data in Tinybird.
I recommend checking out the &lt;a href=&quot;https://www.tinybird.co/docs/guides/deduplication-strategies.html&quot;&gt;Tinybird guide on de-duplication strategies&lt;/a&gt; for the full picture.&lt;/p&gt;

&lt;p&gt;A simple query that only returns the latest items can look like this:&lt;/p&gt;

&lt;div class=&quot;language-sql highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;k&quot;&gt;SELECT&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;FROM&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;task_list&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;ORDER&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;BY&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;last_edited_time&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;DESC&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;LIMIT&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;BY&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;id&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Here’s the effect in Tinybird:&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;/assets/img/blog/2022-10-17-syncing-notion-database-into-tinybird-with-notion-objects/tinybird-query.png&quot; data-toggle=&quot;lightbox&quot; data-gallery=&quot;blog-gallery&quot;&gt;&lt;img src=&quot;/assets/img/blog/2022-10-17-syncing-notion-database-into-tinybird-with-notion-objects/tinybird-query.png&quot; alt=&quot;De-duplicating data in Notion queries&quot; class=&quot;img-blog&quot; /&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2 id=&quot;final-python-program&quot;&gt;Final Python program&lt;/h2&gt;

&lt;p&gt;Here’s the complete Python program&lt;/p&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;kn&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;nn&quot;&gt;re&lt;/span&gt;

&lt;span class=&quot;kn&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;nn&quot;&gt;requests&lt;/span&gt;
&lt;span class=&quot;kn&quot;&gt;from&lt;/span&gt; &lt;span class=&quot;nn&quot;&gt;notion_client&lt;/span&gt; &lt;span class=&quot;kn&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Client&lt;/span&gt;

&lt;span class=&quot;kn&quot;&gt;from&lt;/span&gt; &lt;span class=&quot;nn&quot;&gt;notion_objects&lt;/span&gt; &lt;span class=&quot;kn&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Database&lt;/span&gt;


&lt;span class=&quot;k&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;main&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;():&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;notion_token&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;secret_&quot;&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;tinybird_token&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;p.&quot;&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;database_id&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;&quot;&lt;/span&gt;

    &lt;span class=&quot;c1&quot;&gt;# create the notion client
&lt;/span&gt;    &lt;span class=&quot;n&quot;&gt;client&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Client&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;auth&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;notion_token&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;c1&quot;&gt;# the notion-objects database
&lt;/span&gt;    &lt;span class=&quot;n&quot;&gt;db&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Database&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;database_id&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;client&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;c1&quot;&gt;# convert the title of the notion database into a valid tinybird datasource name
&lt;/span&gt;    &lt;span class=&quot;n&quot;&gt;tinybird_datasource&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;re&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;sub&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;'[^0-9a-zA-Z]+'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;'_'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;db&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;title&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;strip&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()).&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;lower&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt;

    &lt;span class=&quot;n&quot;&gt;r&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;requests&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;get&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;
        &lt;span class=&quot;s&quot;&gt;f'https://api.tinybird.co/v0/sql'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;params&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
            &lt;span class=&quot;s&quot;&gt;'q'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;f&quot;SELECT max(last_edited_time) FROM &lt;/span&gt;&lt;span class=&quot;si&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;tinybird_datasource&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
            &lt;span class=&quot;s&quot;&gt;'token'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tinybird_token&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
    &lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;not&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;r&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;ok&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;or&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;not&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;r&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;text&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;strip&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;():&lt;/span&gt;
        &lt;span class=&quot;c1&quot;&gt;# probably the table was not found, so we just use an empty filter
&lt;/span&gt;        &lt;span class=&quot;n&quot;&gt;query&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{}&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;else&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
        &lt;span class=&quot;c1&quot;&gt;# build a query that fetches only those pages that were edited
&lt;/span&gt;        &lt;span class=&quot;c1&quot;&gt;# since the last time the database was synced
&lt;/span&gt;        &lt;span class=&quot;n&quot;&gt;query&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
            &lt;span class=&quot;s&quot;&gt;&quot;filter&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
                &lt;span class=&quot;s&quot;&gt;&quot;timestamp&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;last_edited_time&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
                &lt;span class=&quot;s&quot;&gt;&quot;last_edited_time&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;after&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;r&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;text&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;strip&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()},&lt;/span&gt;
            &lt;span class=&quot;p&quot;&gt;},&lt;/span&gt;
        &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;

    &lt;span class=&quot;c1&quot;&gt;# build an ndjson batch that we later sync into tinybird
&lt;/span&gt;    &lt;span class=&quot;n&quot;&gt;batch&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;[]&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;record&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;db&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;query&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;query&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;):&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;batch&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;append&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;record&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;to_json&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;flat&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;True&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;))&lt;/span&gt;

    &lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;not&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;batch&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
        &lt;span class=&quot;k&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;batch is empty, probably no data was edited&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
        &lt;span class=&quot;k&quot;&gt;return&lt;/span&gt;

    &lt;span class=&quot;n&quot;&gt;data&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;se&quot;&gt;\n&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;join&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;batch&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;c1&quot;&gt;# post the batch into tinybird and create the table if it doesn't exist
&lt;/span&gt;    &lt;span class=&quot;n&quot;&gt;r&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;requests&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;post&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;
        &lt;span class=&quot;s&quot;&gt;'https://api.tinybird.co/v0/events'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;params&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
            &lt;span class=&quot;s&quot;&gt;'name'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tinybird_datasource&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
            &lt;span class=&quot;s&quot;&gt;'token'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tinybird_token&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;p&quot;&gt;},&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;data&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;data&lt;/span&gt;
    &lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;k&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;r&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;status_code&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;r&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;text&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;


&lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;__name__&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;==&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;'__main__'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;main&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;final-remarks&quot;&gt;Final remarks&lt;/h2&gt;

&lt;p&gt;Ideally, Tinybird partners up with Notion to provide this as a connector.
Until then, we’ll need our own serverless solutions to operationalize this.&lt;/p&gt;

&lt;p&gt;We use similar code at LocalStack in AWS Lambda functions that do the synchronization on a schedule.
You could easily package this as a Lambda function and just deploy it to AWS to do the synchronization when the lambda is invoked.
In fact, when I developed the deployment, I tested my Lambda function on LocalStack locally first.
Look out for a follow-up on our &lt;a href=&quot;https://localstack.cloud/blog&quot;&gt;LocalStack blog&lt;/a&gt; explaining how that works.&lt;/p&gt;

&lt;p&gt;notion-objects is still under development and the API may change drastically!
The next iteration will be allowing update operations on Notion objects (already works for simple properties in notion-objects&amp;gt;=0.4.0), to write back into Notion databases.&lt;/p&gt;

&lt;p&gt;What we haven’t covered so far are page deletions.
With the current script, we cannot remove pages from the Tinybird database - it’s append only.&lt;/p&gt;

&lt;p&gt;Stay tuned for more!&lt;/p&gt;</content><author><name></name></author><category term="HowTo" /><summary type="html">In this blog post, I want to show you how you can, with just a few lines of Python code using the notion-objects library and the Tinybird API, synchronize data from Notion databases into Tinybird data sources.</summary></entry><entry><title type="html">Streaming Lectures at TUW using OBS and YouTube Live</title><link href="/blog/streaming-lectures-with-obs-and-youtube-20200925.html" rel="alternate" type="text/html" title="Streaming Lectures at TUW using OBS and YouTube Live" /><published>2020-09-25T08:00:00-05:00</published><updated>2020-09-25T08:00:00-05:00</updated><id>/blog/streaming-lectures-with-obs-and-youtube</id><content type="html" xml:base="/blog/streaming-lectures-with-obs-and-youtube-20200925.html">&lt;p&gt;During the summer term 2020, the first COVID-19 related shutdown of universities caught us a bit off-guard.
I was teaching Distributed Systems Technologies at TU Wien and had to postpone the first lecture.
It was unclear what the TUW standards for distance learning would be.
The common practice at the time were recorded slidecasts, which I didn’t find to be very engaging.
So we decided to instead stream our lectures at the dates we had scheduled to YouTube live and then provide recordings of them.
Because the model was successful, and I’ve been asked a few times about the setup I used, here’s a short post about it.&lt;/p&gt;

&lt;center&gt;&lt;blockquote class=&quot;twitter-tweet&quot;&gt;&lt;p lang=&quot;en&quot; dir=&quot;ltr&quot;&gt;Successfully streamed the first DST lecture &lt;a href=&quot;https://twitter.com/tuvienna?ref_src=twsrc%5Etfw&quot;&gt;@tuvienna&lt;/a&gt; &lt;a href=&quot;https://twitter.com/dsg_tuwien?ref_src=twsrc%5Etfw&quot;&gt;@dsg_tuwien&lt;/a&gt; from home, and did a live demo of Murphy&amp;#39;s Law: battery of microphone died, battery of camera died... Students were still engaged with the format, and I&amp;#39;m totally looking forward to the next lecture. 👨‍🏫😎 &lt;a href=&quot;https://t.co/8nXoBzPUPc&quot;&gt;pic.twitter.com/8nXoBzPUPc&lt;/a&gt;&lt;/p&gt;&amp;mdash; Thomas Rausch (@thrauat) &lt;a href=&quot;https://twitter.com/thrauat/status/1240296059928182791?ref_src=twsrc%5Etfw&quot;&gt;March 18, 2020&lt;/a&gt;&lt;/blockquote&gt; &lt;script async=&quot;&quot; src=&quot;https://platform.twitter.com/widgets.js&quot; charset=&quot;utf-8&quot;&gt;&lt;/script&gt; &lt;/center&gt;

&lt;h2 id=&quot;video--audio-setup&quot;&gt;Video &amp;amp; Audio Setup&lt;/h2&gt;

&lt;h3 id=&quot;hardware&quot;&gt;Hardware&lt;/h3&gt;

&lt;p&gt;I don’t use any special hardware:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Laptop&lt;/li&gt;
  &lt;li&gt;HD Webcam (&lt;a href=&quot;https://www.logitech.com/en-us/product/hd-pro-webcam-c920&quot;&gt;Logitech C920&lt;/a&gt;)&lt;/li&gt;
  &lt;li&gt;Cheap lavalier microphone (15 EUR on Amazon)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I used my Sony a6000 as a webcam at first (you can see it in the picture on Twitter), but the quality wasn’t that much better, so I went back to the webcam because it was more convenient.&lt;/p&gt;

&lt;h3 id=&quot;software&quot;&gt;Software&lt;/h3&gt;

&lt;p&gt;The only software I used was:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;a href=&quot;https://obsproject.com/&quot;&gt;OBS Studio&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;LibreOffice Impress&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;obs-configuration&quot;&gt;OBS Configuration&lt;/h2&gt;

&lt;p&gt;OBS main concept is a scene composed of sources.
I needed to capture my LibreOffice full-screen presentation window, as well as my webcam.
OBS also allows you to place static text and images in the scene (like the university logo and the course name).&lt;/p&gt;

&lt;h3 id=&quot;presentation&quot;&gt;Presentation&lt;/h3&gt;

&lt;p&gt;To capture the presentation window, I opened the LibreOffice presentation, entered presentation mode (F5), alt-tabbed into OBS, and then added a “Window Capture” source of the full-screen presentation window now running in the background.
You could do it with a complete screen capture, but it didn’t work properly with my multi-monitor setup.
When streaming, I simply switched backed to the presentation window and tried to give a natural presentation as I would in the lecture hall.&lt;/p&gt;

&lt;p&gt;For me that also meant &lt;em&gt;standing&lt;/em&gt; while streaming so the delivery would feel more natural to me.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;/assets/img/blog/2020-09-30-obs-window-capture.jpg&quot; data-toggle=&quot;lightbox&quot; data-gallery=&quot;blog-gallery&quot;&gt;&lt;img src=&quot;/assets/img/blog/2020-09-30-obs-window-capture.jpg&quot; alt=&quot;image&quot; class=&quot;img-blog&quot; /&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3 id=&quot;webcam&quot;&gt;Webcam&lt;/h3&gt;

&lt;p&gt;I oriented my Webcam in a way that I appeared vertically next to the presentation.
I got the idea from &lt;a href=&quot;https://www.youtube.com/watch?v=GyCL8AJUhww&quot;&gt;“goto; conference” recordings&lt;/a&gt;.
Just create a new video capture source, place the source somewhere in the scene, right click and select “Transform &amp;gt; Edit Transform”.
Then just crop and position until it nicely lines up with your presentation source.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;/assets/img/blog/2020-09-17-obs-config.jpeg&quot; data-toggle=&quot;lightbox&quot; data-gallery=&quot;blog-gallery&quot;&gt;&lt;img src=&quot;/assets/img/blog/2020-09-17-obs-config.jpeg&quot; alt=&quot;OBS webcam source&quot; class=&quot;img-blog&quot; /&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2 id=&quot;streaming&quot;&gt;Streaming&lt;/h2&gt;

&lt;p&gt;There are lots of tutorials on how to stream from OBS to YouTube Live, &lt;a href=&quot;https://www.digitaltrends.com/computing/how-to-live-stream-on-youtube-with-obs/&quot;&gt;this one for example&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Basically, after activating the live stream feature in YouTube (takes 24 hours), you schedule a live stream in your YouTube Live studio, find the “Streaming Key”, which you then enter in your OBS configuration.
When you press “start streaming” you should see the preview in YouTube studio.
Once you are ready, you can press “Go Live” in YouTube.&lt;/p&gt;

&lt;p&gt;My streams as well as the recordings were “unlisted”, meaning you can access them with the link, but they cannot be accessed via the YouTube search or video suggestions.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;/assets/img/blog/2020-09-30-obs-yt-live-streaming.png&quot; data-toggle=&quot;lightbox&quot; data-gallery=&quot;blog-gallery&quot;&gt;&lt;img src=&quot;/assets/img/blog/2020-09-30-obs-yt-live-streaming.png&quot; alt=&quot;Streaming configuration&quot; class=&quot;img-blog&quot; /&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2 id=&quot;editing&quot;&gt;Editing&lt;/h2&gt;

&lt;p&gt;I didn’t edit much, I only cut out the, short breaks, coughs, and rambling I sometimes tend to when I don’t have a script.
The YouTube studio editor is absolutely sufficient for that. No editing software needed.
It takes an hour or so after you are down until the edited video appears.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;/assets/img/blog/2020-04-22-youtube-editor.png&quot; data-toggle=&quot;lightbox&quot; data-gallery=&quot;blog-gallery&quot;&gt;&lt;img src=&quot;/assets/img/blog/2020-04-22-youtube-editor.png&quot; alt=&quot;Edit recordings in YouTube editor&quot; class=&quot;img-blog&quot; /&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3 id=&quot;timestamps&quot;&gt;Timestamps&lt;/h3&gt;

&lt;p&gt;Adding timestamps to the video description is a bit of effort, but they are very useful for viewers to navigate the recording!&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;/assets/img/blog/2020-09-30-adding-timestamps.png&quot; data-toggle=&quot;lightbox&quot; data-gallery=&quot;blog-gallery&quot;&gt;&lt;img src=&quot;/assets/img/blog/2020-09-30-adding-timestamps.png&quot; alt=&quot;Creating timestamps&quot; class=&quot;img-blog&quot; /&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;When they are in the video description with a bit of text, YouTube automatically creates chapters in the video progress bar.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;/assets/img/blog/2020-09-30-timestamping-chapters.jpg&quot; data-toggle=&quot;lightbox&quot; data-gallery=&quot;blog-gallery&quot;&gt;&lt;img src=&quot;/assets/img/blog/2020-09-30-timestamping-chapters.jpg&quot; alt=&quot;Timestamp chapters&quot; class=&quot;img-blog&quot; /&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2 id=&quot;conclusion&quot;&gt;Conclusion&lt;/h2&gt;
&lt;p&gt;Overall OBS and YouTube Live make it extremely easy to stream lectures and provide recordings.
For audience participation, I used the live chat, but a lot of lecturers already make use of richer online participation tools.&lt;/p&gt;

&lt;p&gt;By now many lecturers have to do online lectures in some way, and have found a concept that works for them.
The key points that made the lectures work for me were:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Giving a &lt;em&gt;live&lt;/em&gt; lecture at scheduled dates &lt;em&gt;and&lt;/em&gt; provide recordings, rather than pre-record the lecture. &lt;a href=&quot;https://twitter.com/davidcshepherd/status/1308048976961318914&quot;&gt;Pre-recording is hard&lt;/a&gt;, and live streams are more engaging for students.&lt;/li&gt;
  &lt;li&gt;Standing while giving the lecture.&lt;/li&gt;
  &lt;li&gt;Not using the built-in webcam/laptop microphone&lt;/li&gt;
  &lt;li&gt;Making one pass through the recording to a) create timestamps, b) trim out unnecessary parts, c) reflect on my presentation of the content.&lt;/li&gt;
&lt;/ul&gt;</content><author><name></name></author><category term="University" /><category term="HowTo" /><summary type="html">During the summer term 2020, the first COVID-19 related shutdown of universities caught us a bit off-guard. I was teaching Distributed Systems Technologies at TU Wien and had to postpone the first lecture. It was unclear what the TUW standards for distance learning would be. The common practice at the time were recorded slidecasts, which I didn’t find to be very engaging. So we decided to instead stream our lectures at the dates we had scheduled to YouTube live and then provide recordings of them. Because the model was successful, and I’ve been asked a few times about the setup I used, here’s a short post about it.</summary></entry></feed>