Besom and Smithy4s on AWS - Scala 3 good vibes only

scala3scalasmithy4sawsbesomsmithy

TL;DR

What are we doing and why

This section provides an overview of the challenges that led to this post and the historical context behind the tools I've chosen. It's that part before a recipe where the cook waxes poetic for several pages about how the lemons remind them of their grandma.

In this post we want to build a service using Scala 3, which performs some operations with AWS services, but doesn't use the officially provided AWS SDK for Java. Not only do we want to make a running service, we also want to deploy this service to AWS Fargate, using Besom - a pure Scala 3 SDK for Pulumi.

Now, on the surface, "deploy a Scala 3 service to AWS" doesn't sound like anything that warrants a blog post. There are companies that have been doing exactly that for a long time, and some, completely out of their mind, have been doing that with very first Scala 3 milestone releases.

Therefore, to make ourselves feel just a little bit more than the usual boredom and existential dread, we have to make things a little bit more challenging. Back when I worked with Olivier Mélois, he has been experimenting with generating pure Scala clients for AWS services (at the time, using JSON metadata files from AWS), backed by http4s library for HTTP requests.

As we looked at the stars together, we imagined a better world where we can re-use http4s middleware we've already been building for our production services, with things like tracing, rate limiting, retries, metrics, etc. One HTTP stack to rule them all, instead of having AWS SDK for Java runtime compete for resources with your own HTTP stack. We worked on that primordial library, and even implemented some applications with it, some internal tooling, and an AWS Lambda talking to DynamoDB.

A lot of things happened since then, and AWS have put out Smithy - a general IDL for describing SDKs, the very same they've started using to generate SDK clients for their own services. Olivier went ahead and created Smithy4s which, among other things, implemented a powerful code generator for Smithy-specified services (I even have a blog series about it). I have waited for a long time to revisit the subject of SDK-less interactions with AWS services, and with the Smithy specs for various services now finally published to Maven central, it felt like the right time.

As you will see later, the service part is not complicated at all. So to make things a little more interesting, I decided to revisit the old trauma place – AWS infrastructure. Just the mere thought of deploying anything to AWS evokes nightmares of navigating the AWS Console, runtime errors during API calls, bloody fingers from typing out whitespace in Cloudformation templates, etc.

To somewhat alleviate the traumatic experience, various Infrastructure-as-Code solutions have been popping up over the years. Personally I've only had experience with Terraform, and while conceptually I was on board with the execution model, I felt uneasy about the custom language for specifying infrastructure. The sheer quantity of plugins written in Go we utilised also did not make me feel comfortable, as we had at the time dozens of Scala developers and only 1 person comfortable writing Go, and about 3 people comfortable writing Terraform HCL. This created an artificial bottleneck in our ability to share the burden of infrastructure.

So when I first saw Pulumi, I got reasonably excited - even if I only looked at the Java examples at the time. The tipping point for me happened with the release of Besom, which is a pure Scala 3 implementation of Pulumi SDK (with no dependencies on any other Pulumi SDK, not even Java one). And I believe a part of Besom's own appeal is the fact that by default it recommends usage of Scala CLI, a recent addition to Scala tooling space, – which fits very nicely with Pulumi CLI.

Building the app

We're building a very simple service – it takes some text provided by the user, and computes the sentiment – positive, negative, neutral.

The sentiment detection we will delegate to AWS Comprehend - one of Amazon's thousands of services provided as part of the cloud offering.

The entire app will be written with Scala 3.3.3, and we will be using Typelevel libraries, mainly because the only dialect in which Smithy4s generates SDKs heavily relies on Cats Effect concepts and http4s.

To set the scene, let's flesh out the build files and generate the actual AWS SDK code. We will use SBT as our build tool of choice, mostly because this allowed me to steal the build code straight from the docs. See Appendix for a SBT-less deployment.

Smithy4s AWS client for Comprehend

Smithy4s documentation actually has a dedicated page describing in great detail everything you need to know and put in your build files to generate AWS SDK clients for your application.

So, after setting up the Smithy4s plugin, let's add AWS Comprehend to build.sbt:

smithy4sAwsSpecs ++= Seq(AWS.comprehend)

And that's it! If we run compile, Smithy4s will:

  1. Fetch the dependency containing Smithy specs for AWS Comprehend
  2. Generate all the code in a folder managed by SBT

Here's where I hit my first hurdle – more than 400 files were generated! That's all the operations and their supporting type aliases and data types. What I discovered is that compiling them all from scratch on Scala 3 takes a considerable amount of time. Once incremental compilation kicks in at later stages, this stops being a problem. But because of the way I build and deploy (see below), this initial slow down was unacceptable.

What's more, I suspect that this slow compilation is down to some Scala 3 bug (Discord), and not a fundamental lower bound on compilation speed, because compiling this many files on 2.13 is still not instant, but 5 times faster. I'm sure it will be fixed eventually, but for now I needed a different approach.

After consulting with Smithy4s maintainers, we arrived at a solution – allow user to customise which operations from services to keep in generated code. The effect is especially visible in this small service which only uses 1 operation from AWS Comprehend - DetectSentiment. The way this new mechanism is triggered is by applying a smithy4s.meta#only annotation to operations you'd like to keep. So in our own application we can create a smithy file like this:

$version: "2"

namespace my.code

use smithy4s.meta#only

apply com.amazonaws.comprehend#DetectSentiment @only

This brings 400-ish files down to 16-ish. A great reduction with tremendous effect on JAR size, and compilation speed.

At the time of writing Smithy4s that supports this annotation is released as 0.18.0-12-da4b23a, but hopefully will be soon available in a stable release. You can find more details in the yet unreleased doc page I wrote.

This annotation is available in Smithy4s 0.18.16 and you can read more about it in a dedicated doc page.

I will omit the code for setting up the AWS Comprehend client as it mostly matches what you can find in the Example usage section of the Smithy4s AWS tutorial. The code in the repo is slightly more involved, as I was dealing with some timeouts in accessing credentials from Fargate – there's an opportunity to improve this code and remove the need for my hacks.

For the rest of backend work let's just assume that we have this method implemented:

import com.amazonaws.comprehend.* // Generated code from specs

def comprehendService: Resource[IO, Comprehend[IO]] = // ...

Where Comprehend[IO] is the class containing AWS Comprehend operations.

Backend

The app itself is a HTTP server with two endpoints:

  1. / which should just render a HTML page with the form to submit text
  2. /analyse to which the from will submit a POST request with user text

Let's focus on /analyse for now. We want to accept the submitted form, run some basic validations on the text (shouldn't be empty, shouldn't be too long) then run it through AWS Comprehend and handle the result.

This endpoint will directly return HTML snippets, because we are living our best lives and implementing the frontend with HTMX. These gentle hands shall touch no dirty JSON. All the HTML snippets logic will be confined to Html object, which we will talk about later. To render HTML we're using http4s-scalatags integration.

Here's the implementation of our http4s Routes with comments inline:

private def routes(comprehendService: Comprehend[IO]): HttpApp[IO] =
  // http4s imports
  import org.http4s.*, dsl.io.*, implicits.*
  HttpApp[IO]: // ...
    // index page
    case GET -> Root =>
      // Render index page and nothing else
      Ok(Html.template(Html.userInput))

    // /analyse endpoint
    case request @ POST -> Root / "analyse" =>
      request
        .as[UrlForm]
        .flatMap: form =>
          // get the text parameter, trim it and valiadte
          form.get("text").headOption.map(_.trim).filter(_.nonEmpty) match
            case None =>
              Ok(Html.error("Text cannot be empty"))

            case Some(text) if text.length > 1024 =>
              Ok(Html.error("Text cannot be longer than 1024 characters"))

            case Some(text) =>
              // call AWS Comprehend
              comprehendService
                .detectSentiment(CustomerInputString(text), LanguageCode.EN)
                .attempt // this is to ensure we're explicit about error handling
                .flatMap:
                  case Left(ex) =>
                    log.error("Failed to detect sentiment for text", ex) *>
                      Ok(Html.error("Some internal error has occurred"))
                  case Right(res) =>
                    res.sentiment match
                      case None => Ok(Html.error("No sentiment detected"))
                      case Some(sentiment) => Ok(Html.sentiment(sentiment))

    case _ => NotFound()
 end routes

These routes can be mounted into a running server like this:

private def httpServer: Resource[IO, server.Server] =
  comprehendService
    .map(routes) // sic! invoke our routes(comprehendService) from above
    .flatMap: routes =>
      EmberServerBuilder
        .default[IO]
        .withPort(
          sys.env
            .get("SMITHY4S_PORT") // sic! we will control the binding port from within the container
            .flatMap(Port.fromString)
            .getOrElse(port"8080")
        )
        .withHost(host"0.0.0.0")
        .withHttpApp(routes)
        .withShutdownTimeout(0.second)
        .build

And the entry point to our application then looks something like this:

object Server extends IOApp.Simple:
  def run =
    httpServer.use: server =>
      log.info(s"Running at: ${server.baseUri}") *>
        IO.never

And that's it!

The entire code can be found in App.scala it contains some modifications, but the gist is still the same - nothing conceptually different to what is presented here.

Frontend with HTMX

As mentioned before, we are using HTMX, which is a hypertext frontend "framework" which is quite boastful about its minimalism, urging website developers to go back to their roots – touch the warm grass of hypertext, abstain from JSON and put a cork in that bottle of SPA kool-aid.

While my personal feelings about it are mixed, I believe its usage here is justified:

  1. HTMX is available on a CDN (and in fact CDN is the recommended supplier), meaning no need to mess about with bundlers
  2. There really isn't much interactivity in this app, all I need is something to do AJAX for me without having to even look at Fetch or XMLHttpRequest

We will render all HTML tags from our app using Scalatags, and to make things a bit more Scala idiomatic, we can define a helper like this:

/** Helper to create htmx attributes. `hx.replace` will create a `hx-replace`
  * attribute
  */
object hx extends Dynamic:
  def selectDynamic(name: String): scalatags.Text.Attr =
    scalatags.Text.tags.attr("hx-" + name)

I won't try and do a full HTMX tutorial (given that I skipped that myself), and just show the bits we will need:

form(
  div(
    cls := "flex justify-center items-center",
    textarea(
      placeholder := "type here...",
      name        := "text", // <-- this is the input element which will contain user text
      // ...
    ),
    // ...
    // this button contains HTMX' interactivity attributes
    button(
      hx.post   := "/analyse", // submit the surrounding form with POST method to /analyse
      hx.target := "#result", // place the result from the /analyse call into a HTML element with id "result"
      "Analyse",
      // ...
    )
  // ...  
  div(cls := "text-xl", "Sentiment: ", span(id := "result", "...")) // this is where the result will be placed

All of the HTML code is in the Html object.

Believe it or not, that's the end of the app itself! In the companion repository, you can run sbt reStart in the app/ subfolder, and if you have AWS credentials configured (like for AWS CLI), you too will be able to use AWS Comprehend.

Packaging the app

While we can ship our entire application as a Docker container where we run sbt run, this is quite wasteful. We don't want to compile the application on startup, and we don't need all those SBT dependencies to run the service.

So we will use my favourite technique for shipping containers – multi-stage docker builds.

The general idea is that we compile our app from scratch in one container, and then only copy the runnable artefacts into another container, more optimised for runtime performance. Building the app from scratch inside the container can also give us reproducibility and simplify getting started experience.

You can see the entire Dockerfile in the companion repo, but here's a heavily commented version of it:

FROM eclipse-temurin:21 as builder

# Wget is needed to bootstrap SBT
RUN apt update && apt install wget

WORKDIR /app

# bootstrap SBT
RUN wget https://raw.githubusercontent.com/sbt/sbt/1.10.x/sbt && chmod +x ./sbt
# we only copy the single file that affects which version of SBT gets bootstrapped
# This allows us to cache these steps if you don't change SBT version
COPY project/build.properties project/build.properties
# Start SBT with a dummy command so that it can pre-fetch all the dependencies required
RUN ./sbt --sbt-create version

# build app
# We copy all the application sources, so the building step will 
# run if there are any modifications
COPY build.sbt build.sbt
COPY project/plugins.sbt project/plugins.sbt
COPY App.scala App.scala
# this will produce a runnable package in /app/target/universal/stage
RUN ./sbt stage

# this is the start of the runtime container - one we will actually ship to AWS
# we're using GraalVM, which generally performs better than other JDK implementations
# and we're also using JDK 22, latest available version at the time of this blogpost
FROM ghcr.io/graalvm/jdk-community:22
RUN mkdir -p /opt
# copy just the built app folder into this container - it doesn't need SBT to run,
# only JVM
COPY --from=builder /app/target/universal/stage /opt/app

ENV SMITHY4S_PORT=80

EXPOSE 80

# run the app
CMD ["/opt/app/bin/app"]

This Dockerfile is self-contained, and you can build the app from scratch by just running docker build . -t besom-smithy4s. Self-contained docker build plays particularly well with Pulumi's Docker provider, which provides a convenient way to build a docker image and use it for deployment with any providers that support it.

Deploying with Besom

Now it's time to actually put our service out there for public consumption. We will be deploying to AWS Fargate.

First thing's first, I set up a Besom project in besom folder of my repo, using instructions from the documentation. At the end of it, you should have pulumi CLI available, and Besom runtime registered with Pulumi.

The project that is being set up is a standard Scala CLI project, and Metals should work with it correctly straight away. Note that the default template enables a Besom compiler plugin, and, as it turns out, there's a potential issue in Scala CLI and Metals interaction. I've disabled the compiler plugin until the issue is fixed, and can't say it had any impact on my experience with Besom, given how small the infra is.

At this point we are ready to write our infrastructure code. Let's go over the parts of the infrastructure we need to create:

  1. An AWS ECR repository created, where we can push our Docker images

  2. An ECR image pushed to the repo

  3. A VPC for public and private networking

    Note that AWS has a default VPC available on all accounts, but it seems I've accidentally deleted it at some point, and it cannot be restored. This means we have to create a VPC from scratch, but Pulumi does allow you to retrieve a default VPC and reference that instead. If you have it, of course.

  4. In ECS:

    1. Cluster
    2. Task definition
    3. Service that references the task definition

  5. IAM Policy attachment

    Note because our service needs to access AWS Comprehend, we explicitly allow this access by attaching a policy to the ECS Task role that is created automatically for us.

  6. Optionally - a named log group in Cloudwatch for our service

One thing that wasn't immediately clear to me when looking at Besom examples for AWS, is that packages and definitions seem to be divided between aws and awsx, e.g.:

//> using scala "3.3.3"
//> using options -Werror -Wunused:all -Wvalue-discard -Wnonunit-statement
//> using dep "org.virtuslab::besom-core:0.2.2"
//> using dep "org.virtuslab::besom-aws:6.23.0-core.0.2"
//> using dep "org.virtuslab::besom-awsx:2.5.0-core.0.2"

awsx refers to Pulumi AWS Crosswalk – pre-built Pulumi components for common pieces of infrastructure on AWS, which follow best practices. For example awsx contains a ecr.Repository component, which under the hood is actually a aws.ecr.Repository and a lifecycle policy that is pre-configured for you:

    ├─ awsx:ecr:Repository                                              sentiment-service-repo
    │  ├─ aws:ecr/repository:Repository                                 sentiment-service-repo
    │  └─ aws:ecr/lifecyclePolicy:LifecyclePolicy                       sentiment-service-repo

In my experiments I was unlucky enough to hit on what appears to be an issue in Pulumi itself – so to work around it, I decided to use both the raw definitions, and definitions from AWS Crosswalk.

Pulumi is declarative – your runtime of choice will formulate a plan for resources to be built, and then Pulumi state management logic will come up with an execution plan to actually run requests against real infrastructure, e.g. invoking AWS APIs.

To make our Scala CLI script into something Pulumi can invoke and interpret the results from, we just need to use Pulumi.run function provided by Besom, and make sure we return all the infrastructure resources as part of Besom's Stack. Examples should make it clear.

Let's start with the ECR image repository:

@main def main =
  Pulumi.run:
    val repository =
      awsx.ecr.Repository(
        "sentiment-service-repo",
        awsx.ecr.RepositoryArgs(forceDelete = true)
      )

Here's we're using AWS Crosswalk component – I fully qualified the package names to make it clear, but it's not necessary.

Next, we define our image like so:

val image = awsx.ecr.Image(
  "sentiment-service-image",
  awsx.ecr.ImageArgs(
    repositoryUrl = repository.url,
    context = p"../app",
    platform = "linux/amd64"
  )
)

This is where the convenience of Pulumi's built-in components starts to help us tremendously – by just specifying the context parameter to point at the relative path to your application, we instruct Pulumi to build the Docker image in that folder. Because our Dockerfile is self-contained, we don't need to add any other flags or parameters, or introduce any extra steps.

Next let's create the VPC and the load balancer referencing the VPC's public subnets:

val vpc = awsx.ec2.Vpc("sentiment-service-vpc")

val loadBalancer = awsx.lb.ApplicationLoadBalancer(
  "sentiment-service-lb",
  ApplicationLoadBalancerArgs(subnetIds = vpc.publicSubnetIds)
)

Note that we are using AWS Crosswalk VPC – and as it turns out, the best practices it follow require the setup of 3 NAT Gateways, along with a ton of other infrastructure (I provide the full Pulumi stack plan below). The reason I'm mentioning this is because NAT Gateways are expensive (in hobbyist terms) and cost me 40$ over the course of several days I had this service running. Very unpleasant surprise.

The most involved resources are ECR task definition (which describes our container runtime) and the ECR Service (which describes a fleet of tasks as a single entity). These two definitions caused me the most amount of pain, at first due to the bug in Pulumi itself, and then due to the peculiarities of AWS runtime constraints that aren't really encoded in the API definitions – some things you just have to know. That knowledge has been wiped from my memory after not using AWS for a few years. So I had to work through multiple failures from AWS, tweaking the parameters here and there, until the stack finally deployed cleanly. I had to solicit help from my ex-colleague who had all the AWS knowledge etched deep into his cortex.

Here's the task definition:

val task = awsx.ecs.FargateTaskDefinition(
  "sentiment-service-task",
  FargateTaskDefinitionArgs(
    containers = Map(
      "sentiment-service" -> TaskDefinitionContainerDefinitionArgs(
        image = image.imageUri,
        name = "sentiment-service",
        cpu = 128,
        memory = 512,
        essential = true,
        logConfiguration = TaskDefinitionLogConfigurationArgs(
          logDriver = "awslogs",
          options = JsObject(
            "awslogs-group"         -> JsString("sentiment-service-logs"),
            "awslogs-region"        -> JsString("us-east-1"),
            "awslogs-stream-prefix" -> JsString("ecs")
          )
        ),
        portMappings = List(
          TaskDefinitionPortMappingArgs(
            containerPort = 80
          )
        )
      )
    )
  )
)

Note that we are referencing image in our task definition – Besom has many clever type-level tricks to make writing these definitions pleasant – and to support declarative model of programming, ensuring that once image resource is actually created in AWS, its imageUri property will be available to task resource creation.

To make sure our ECS task can access AWS Comprehend, we create a policy attachment:

val policyAttachment = aws.iam.RolePolicyAttachment(
  "sentiment-service-task-comprehend-policy",
  RolePolicyAttachmentArgs(
    role = task.taskRole.map(_.get),
    policyArn = "arn:aws:iam::aws:policy/ComprehendReadOnly" // this is the policy ARN that allows Comprehend access
  )
)

And finally, we define a ECS Service:

val service = aws.ecs.Service(
  "sentiment-service",
  aws.ecs.ServiceArgs(
    launchType = "FARGATE",
    taskDefinition = task.taskDefinition.arn,
    cluster = cluster.arn,
    desiredCount = 1,
    networkConfiguration = ServiceNetworkConfigurationArgs(
      subnets = vpc.publicSubnetIds,
      assignPublicIp = true,
      securityGroups =
        loadBalancer.defaultSecurityGroup.map(_.map(_.id)).map(_.toList)
    ),
    loadBalancers = List(
      ServiceLoadBalancerArgs(
        containerName = "sentiment-service",
        containerPort = 80,
        targetGroupArn = loadBalancer.defaultTargetGroup.arn
      )
    )
  )
)

Note that we're using a bare aws definition of Service - I've had some issues with using the AWS Crosswalk FargateService and through several iterations of fighting with AWS itself, I arrived at using the raw service, and manually wiring in our VPC, Load Balancer, and cluster definitions.

At the very end of our deployment script, we produce the full Stack we want Pulumi to instantiate:

Stack(repository, logGroup, service, cluster, policyAttachment).exports(
  image = image.imageUri,
  url = p"http://${loadBalancer.loadBalancer.dnsName}"
)

When the stack is deployed, Pulumi will output image and url with instantiated resources for our convenience.

You can deploy the entire stack yourself by running cd besom && pulumi up -s dev in the companion repository.

Final state of the Pulumi stack along with the outputs 
    Owner: velvetbaldmime
    Last updated: 22 hours ago (2024-04-06 10:19:08.715013 +0100 BST)
    Pulumi version used: v3.112.0
Current stack resources (59):
    TYPE                                                                NAME
    pulumi:pulumi:Stack                                                 besom-smithy4s-kiss-dev
    ├─ aws:cloudwatch/logGroup:LogGroup                                 sentiment-service-log-group
    ├─ aws:ecs/cluster:Cluster                                          sentiment-service-cluster
    ├─ awsx:ec2:Vpc                                                     sentiment-service-vpc
    │  └─ aws:ec2/vpc:Vpc                                               sentiment-service-vpc
    │     ├─ aws:ec2/internetGateway:InternetGateway                    sentiment-service-vpc
    │     ├─ aws:ec2/subnet:Subnet                                      sentiment-service-vpc-private-2
    │     │  └─ aws:ec2/routeTable:RouteTable                           sentiment-service-vpc-private-2
    │     │     ├─ aws:ec2/routeTableAssociation:RouteTableAssociation  sentiment-service-vpc-private-2
    │     │     └─ aws:ec2/route:Route                                  sentiment-service-vpc-private-2
    │     ├─ aws:ec2/subnet:Subnet                                      sentiment-service-vpc-private-3
    │     │  └─ aws:ec2/routeTable:RouteTable                           sentiment-service-vpc-private-3
    │     │     ├─ aws:ec2/routeTableAssociation:RouteTableAssociation  sentiment-service-vpc-private-3
    │     │     └─ aws:ec2/route:Route                                  sentiment-service-vpc-private-3
    │     ├─ aws:ec2/subnet:Subnet                                      sentiment-service-vpc-public-3
    │     │  ├─ aws:ec2/routeTable:RouteTable                           sentiment-service-vpc-public-3
    │     │  │  ├─ aws:ec2/routeTableAssociation:RouteTableAssociation  sentiment-service-vpc-public-3
    │     │  │  └─ aws:ec2/route:Route                                  sentiment-service-vpc-public-3
    │     │  ├─ aws:ec2/eip:Eip                                         sentiment-service-vpc-3
    │     │  └─ aws:ec2/natGateway:NatGateway                           sentiment-service-vpc-3
    │     ├─ aws:ec2/subnet:Subnet                                      sentiment-service-vpc-private-1
    │     │  └─ aws:ec2/routeTable:RouteTable                           sentiment-service-vpc-private-1
    │     │     ├─ aws:ec2/routeTableAssociation:RouteTableAssociation  sentiment-service-vpc-private-1
    │     │     └─ aws:ec2/route:Route                                  sentiment-service-vpc-private-1
    │     ├─ aws:ec2/subnet:Subnet                                      sentiment-service-vpc-public-2
    │     │  ├─ aws:ec2/routeTable:RouteTable                           sentiment-service-vpc-public-2
    │     │  │  ├─ aws:ec2/route:Route                                  sentiment-service-vpc-public-2
    │     │  │  └─ aws:ec2/routeTableAssociation:RouteTableAssociation  sentiment-service-vpc-public-2
    │     │  ├─ aws:ec2/eip:Eip                                         sentiment-service-vpc-2
    │     │  └─ aws:ec2/natGateway:NatGateway                           sentiment-service-vpc-2
    │     └─ aws:ec2/subnet:Subnet                                      sentiment-service-vpc-public-1
    │        ├─ aws:ec2/eip:Eip                                         sentiment-service-vpc-1
    │        ├─ aws:ec2/routeTable:RouteTable                           sentiment-service-vpc-public-1
    │        │  ├─ aws:ec2/route:Route                                  sentiment-service-vpc-public-1
    │        │  └─ aws:ec2/routeTableAssociation:RouteTableAssociation  sentiment-service-vpc-public-1
    │        └─ aws:ec2/natGateway:NatGateway                           sentiment-service-vpc-1
    ├─ awsx:lb:ApplicationLoadBalancer                                  sentiment-service-lb
    │  ├─ aws:lb/targetGroup:TargetGroup                                sentiment-service-lb
    │  ├─ aws:ec2/securityGroup:SecurityGroup                           sentiment-service-lb
    │  ├─ aws:lb/loadBalancer:LoadBalancer                              sentiment-service-lb
    │  └─ aws:lb/listener:Listener                                      sentiment-service-lb-0
    ├─ awsx:ecr:Repository                                              sentiment-service-repo
    │  ├─ aws:ecr/repository:Repository                                 sentiment-service-repo
    │  └─ aws:ecr/lifecyclePolicy:LifecyclePolicy                       sentiment-service-repo
    ├─ awsx:ecr:Image                                                   sentiment-service-image
    │  └─ docker:index/image:Image                                      14ab35b3-container
    ├─ awsx:ecs:FargateTaskDefinition                                   sentiment-service-task
    │  ├─ aws:iam/role:Role                                             sentiment-service-task-execution
    │  ├─ aws:cloudwatch/logGroup:LogGroup                              sentiment-service-task
    │  ├─ aws:iam/role:Role                                             sentiment-service-task-task
    │  ├─ aws:iam/rolePolicyAttachment:RolePolicyAttachment             sentiment-service-task-execution-9a42f520
    │  └─ aws:ecs/taskDefinition:TaskDefinition                         sentiment-service-task
    ├─ aws:iam/rolePolicyAttachment:RolePolicyAttachment                sentiment-service-task-comprehend-policy
    ├─ aws:ecs/service:Service                                          sentiment-service
    ├─ pulumi:providers:aws                                             default
    ├─ pulumi:providers:awsx                                            default
    ├─ pulumi:providers:aws                                             default_6_9_0
    ├─ pulumi:providers:pulumi                                          default
    └─ pulumi:providers:docker                                          default_4_5_0
Current stack outputs (2):
    OUTPUT  VALUE
    image   579742272793.dkr.ecr.us-east-1.amazonaws.com/sentiment-service-repo-53b3bec@sha256:c78976ff494d33d97cc06f57365fce88fe2c72b4d83e9e234ce82c7412d6b677
    url     http://sentiment-service-lb-64ed2b0-1096276818.us-east-1.elb.amazonaws.com

More information at: https://app.pulumi.com/velvetbaldmime/besom-smithy4s-kiss/dev


Use pulumi stack select to change stack; pulumi stack ls lists known ones

Conclusion

Assessing my experience as a whole would be difficult, because of a big complex (and orange) elephant in the room – AWS. I was done with the service in a fraction of a time it took me to understand all the intricacies of VPCs, subnets, load balancers, etc. that make AWS tick.

I wasted many hours being unaware that pulling an image from ECR constitutes a public network request, and therefore the networking has to be configured even for that operation, which I considered fully internal.

That said, once you remove the AWS-specific part of this exercise, I enjoyed Besom – I was able to translate examples from Pulumi documentation pretty painlessly, the compiler did help me quite a bit in navigating the data structures and Besom's API. There are still rough edges, of course, but they're being addressed, and still being in early stages, it already delivers very good coverage of Pulumi components. So far I have been enjoying the encoding Besom utilises to represent the asynchronous nature of Pulumi declarations. And at no point was I under the impression that I'm writing anything else other than Scala 3, which is a massive win in my book. The fact that Scala CLI allows for a seamless CLI experience with Pulumi also makes Besom feel as a natural, properly supported SDK.

The smithy4s part, frankly, was by far the easiest. The code generation was solid, examples from documentation Just Worked™️, and despite a few rough edges in the supporting runtime, it was a flawless experience. Of course using AWS in this way is still early and experimental, and cuts might occur when walking on the bleeding edge. Some AWS operations are not supported (ones that require streaming, most notably fundamental S3 operations), but they will be in the future. And the runtime already supports Scala.js and Scala Native, meaning it can become a viable option in AWS Lambdas and in smaller, leaner containers.

Appendix: removing SBT from the equation

Despite this post was getting already about 3 times longer than I hoped, I decided to torture myself one more time – and convert the build to Scala CLI, removing SBT entirely. This code lives on a separate scala-cli branch of the companion repo, and all the changes are summarised in this commit.

One caveat is that this removes our ability to generate code on demand – Scala CLI has no support for code generators (yet). So we will need to find a way to pre-generate the code and check it in. Thankfully, Smithy4s ships with a CLI.

Let's install a particular version of this CLI with coursier:

$ cs install --channel https://disneystreaming.github.io/coursier.json smithy4s:0.18.16

Then, all we need is to point this CLI at our Smithy file (with @only annotations), and add a dependency on the AWS spec we require – the CLI will resolve dependencies automatically:

$ smithy4s generate --dependencies com.disneystreaming.smithy:aws-comprehend-spec:2023.09.22 only.smithy --output aws-generated/

This will generate all the Scala files in aws-generated/ folder.

After that, let's remove all the SBT build files and convert the dependencies to Scala CLI format, putting them directly into App.scala:

//> using scala 3.3.3
//> using dep com.disneystreaming.smithy4s::smithy4s-aws-http4s::0.18.16
//> using dep com.disneystreaming.smithy4s::smithy4s-http4s::0.18.16
//> using dep org.http4s::http4s-ember-server::0.23.26
//> using dep org.http4s::http4s-ember-client::0.23.26
//> using dep org.http4s::http4s-scalatags::0.25.2
//> using dep com.outr::scribe-cats::3.13.2
//> using dep com.outr::scribe-json-circe::3.13.2
//> using dep com.outr::scribe-slf4j::3.13.2
//> using dep com.lihaoyi::scalatags::0.12.0

After this, we can easily package our application by calling Scala CLI:

$ scala-cli --power package --assembly App.scala aws-generated -f -o ./assembly

And this will produce a runnable fat JAR.

To finish, we need to modify our Dockerfile as well – first, change SBT bootstrap to bootstrapping Scala CLI:

# bootstrap Scala CLI
RUN wget https://raw.githubusercontent.com/VirtusLab/scala-cli/main/scala-cli.sh && mv scala-cli.sh /usr/bin/scala-cli && chmod +x /usr/bin/scala-cli
# Start Scala CLI with a dummy command so that it can pre-fetch all the dependencies required
RUN scala-cli version

I'm intentionally not using the Scala CLI provided Docker image because it's only published for linux/amd64 platform, which makes it very slow on my Apple Silicon machine (using arm64 architecture).

The app building part needs to be modified as well:

# build app
# We copy all the application sources, so the building step will 
# run if there are any modifications
COPY App.scala App.scala
COPY aws-generated aws-generated
COPY Makefile Makefile
RUN make assembly

Running the app is very similar:

# copy just the built app folder into this container - it doesn't need Scala CLI to run,
# only JVM
COPY --from=builder /app/assembly /opt/app/assembly

ENV SMITHY4S_PORT=80
EXPOSE 80

# run the app
CMD ["/opt/app/assembly"]

And that's it! We don't need to change anything in our infra code, as the Docker image is still built using the same docker build command.

  1. note that this costs actual money in NAT Gateway costs so at the time of you reading it I probably have shut it down

    ↩