Designing Data-Intensive Applications – Data Models: Query Languages

Podcast: Play in new window | Download

Subscribe: Apple Podcasts | Spotify | TuneIn | RSS

We dive into declarative vs imperative query languages as we continue to dive into Designing Data-Intensive Applications while Allen is gallivanting around London, Michael had a bullish opinion, and Joe might not know about The Witcher.

If you’re reading this episodes show notes via your podcast player, you can find this episode’s full show notes at https://www.codingblocks.net/episode125 where you can join in on the conversation.

Sponsors

• Datadog – Sign up today at codingblocks.net/datadog for a free 14 day trial and get a free Datadog t-shirt after creating your first dashboard. Read Datadog’s new State of Serverless research report that breaks down the state of Serverless, with a detailed look at AWS Lambda usage.
• Educative.io – Level up your coding skills, quickly and efficiently. Visit educative.io/codingblocks to get 10% off any course or annual subscription.
• Clubhouse – The fast and enjoyable project management platform that breaks down silos and brings teams together to ship value, not features. Sign up to get two additional free months of Clubhouse on any paid plan by visiting clubhouse.io/codingblocks.

Survey Says

News

• We thank everyone that left us some great reviews:
  • iTunes: 3divint, RyansWorld23
  • Stitcher: Thomasvc, thew_s_witcher4, DaveTheShirt, Yarpskendaya
• Get your shin kicking shoes on and sign up for the South Florida Software Developers Conference 2020, February 29th, where Joe will be giving his talk, Streaming Architectures by Example. (fladotnet.com)
• Come meet us at the 15th annual Orlando Code Camp & Tech Conference, March 28th. Grab some swag and kick us in the shins. (orlandocodecamp.com)

Query Languages

Declarative vs Imperative

• The relational model introduced a declarative query language: SQL.
• Prior models used imperative code.
• An imperative language performs certain operations in a certain order, i.e. do this, then do that.
• With a declarative query language, you specify the pattern of data you want, the conditions that must be met, any sorting, grouping, etc.
  • Note that you don’t specify how to retrieve the data. That is left to the optimizer to figure out.
• Declarative languages are attractive because they are shorter and easier to work with.
  • Consider UI frameworks where you declaratively describe the UI without needing to write code that actually draws a button of a specific size in a specific place with a specific label, etc.
• Additionally, declarative languages hide the implementation details.
  • This means it’s easier to continue using the code as the underlying engine is updated, be it a database, UI framework, etc.
  • This also means that the declarative code can take advantage of performance enhancements with little to no change (often) to the declarative code.
• Because declarative languages only specify the result, instead of how to get the result, they are often more likely to be able to take advantage of parallel execution.
  • Conversely, because imperative code needs to happen in a specific order, it’s more difficult to parallelize.

MapReduce

• Made popular by Google, MapReduce is a programming model meant for processing large amounts of data in bulk in a horizontally distributed fashion.
• Some NoSQL databases, such as MongoDB and CouchDB, support MapReduce in a limited form as a way to perform read-only queries across many documents.
• MapReduce isn’t a declarative query language but it’s also not completely an imperative query API either.
  • This is because to use it, you’re implementing the Template Pattern (episode 16).
• With MapReduce, you implement two methods: map() and reduce().
• The map() and reduce() functions are pure functions.
  • They can only use the data passed into them, they can’t perform additional queries, and they must not have side effects.
  • Pure functions are a concept used in functional programming.
• From a usability perspective though, it does require writing two functions that are somewhat tied to each other, which may be more effort than just writing a single SQL query.
  • Plus a purely declarative SQL query is better able to take advantage of the optimizer.
    • For this reason, MongoDB added a declarative query language called the aggregation pipeline to wrap the MapReduce functionality.
      • It’s expessiveness is similar to a subset of SQL but in a JSON syntax.

Graph-Like Data Models

• Relationships, particularly many-to-many, are an important feature for distinguishing between when to use which data model.
• As relationships get even more complicated, graph models start to feel more natural.
• Where as document databases have documents, and relational databases have tables, rows, and columns, graph databases have:
  • Vertices: Nodes in the graph
  • Edges: Define the relationships between nodes, and can contain data about those relationships.
• Examples of graph-like data:
  • Social graphs: Vertices are the entities (people, media, articles), and edges are the relationships (friends with, likes, etc.)
  • Web graph: Vertices are the pages, and edges are the links.
  • Maps: Addresses are the vertices, and roads, rails, sidewalks are the edges.
• There are some things that are trivial to express in a graph query that are really hard any other way.
  • For example, fetch the top 10 people that are friends with my friends, but not friends with me, and liked pages that I like sorted by the count of our common interests.
• These queries work just like graph algorithms, you define how the graph is traversed.
• Graph databases tend to be highly flexible since you can keep adding new vertices and nodes without changing any other relationships.
• This makes graphs great for evolvability.

Resources We Like

• Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems by Martin Kleppmann (Amazon)
• Grokking the System Design Interview (Educative.io)
• Design Patterns Part 2 – Oh behave! (episode 16)
• Developer Survey Results 2019 (Stack Overflow)
• Graph Algorithms (episode 85)
• CloudSQL with Amy Krishnamohan (gcppodcast.com)

Tip of the Week

• Is there an equivalent of tail -f on Windows? (Stack Overflow)
• Recursively find files whose content matches a regex pattern and display the first 10 lines for context:

Get-ChildItem .\*.txt -Recurse | Select-String -Pattern 'MyPattern' -context 10,0

• Use the Microsoft Application Inspector to identify and surface well-known features and other interesting characteristics of a component’s source code to determine what it is and/or what it does. (GitHub)
• Automatically silence those pesky, or worse: embarrassing, notifications while screensharing on your Mac. (Muzzle)