As long as AI serves as a co-pilot rather than an auto-pilot, there's room for a language that facilitates effective collaboration between humans and AI by reducing cognitive load and enabling rapid testing, thus significantly cutting iteration times. Moreover, AI simplifies the adoption of new languages.
Hey everyone!
We're delighted to share with you the third ever issue of the Wing Inflight Magazine (here's a link to our last issue if you missed it).
The Inflight Magazine is where you can stay up to date with Wing Programming Language developments, community events, and all things Wing.
Have comments or questions about Wing? Hit us up at @winglangio on Twitter or leave a message in our Slack!
This week's cloud computing joke...
Q: Why do serverless developers make terrible comedians?
A: Because their jokes always end up with a cold start!
Hi there!
We're excited to share with you the second issue of the Wing Inflight Magazine (here's a link to our first issue if you missed it).
The Inflight Magazine is where you can stay up to date with Wing Programming Language developments and the awesome community that is forming around it.
You received this email because we have you on our list as someone who might be interested to stay informed about the Wing Programming Language.
As always, we would love to hear what you think. Feel free to reply directly to this email, mention @winglangio on Twitter or ping us on slack.
Okay, so you have decided to write your amazing application in Wing. You have enjoyed the benefits of Wing's cloud-oriented programming model and high level abstractions. Everything works great, the queues are queuing, the functions are functional, and the buckets are filling. You are ready to hand this application off to the ops team for deployment when suddenly, you are told there is a problem: the infrastructure doesn't comply with your organizations cloud excellence requirements.
Susan, who is an underappreciated, and sleep deprived platform engineer, tells you that your taggable infra resources must adhere to a rigorous tagging convention. She continues to tell you that all buckets must have versioning and replication enabled. You also gather that she was probably going out for drinks later, since she kept going on about her security group and ciders.
Before you take to Twitter and post a long thread about how Wing is not enterprise ready, you recall the tech lead (we will call him Greg) who gave a presentation about Wing at your organization's last Cloud Center of Excellence (CCoE) meeting. Greg assured everyone that they would be able to use Wing and only focus on the functional aspects of their cloud applications. He said this would be made possible by leveraging the organization's custom Wing plugins. So now all that remains is, to figure out what a Wing plugin is.
The Wing SDK is hard at work abstracting away the non-functional concerns of your cloud application. Which is great, now you can focus on the business logic of your application and not even care about what cloud this code will run on. However, these abstractions only solve a piece of the puzzle that is the cloud compiler. Inevitably with any production grade deployment, we will need a way to customize the compilation output to meet business requirements. Whether they be security, compliance, or cost optimizations these scenarios will require drilling down bellow the abstractions and into the compiler.
This is where the Wing plugin system comes in as the first steps to opening up hooks into the Wing compilation process. By using these plugin hooks Wing is still able to decouple the functional and non-functional concerns of our applications. Think of it as the SDK handles all functional concerns such as queues, functions, and buckets, while the plugin system handles the non-functional concerns such as encryption, versioning, and security groups.
The plugin system is boosting the Wing toolchain into the next level of cloud development. Unlocking the ability for teams to solve complex real world problems in Wing without compromising their organizations cloud principles. Actually, the plugin system enables organizations to double down and enforce their cloud principles without slowing down innovation.
I think the "why" is important to talk about for a moment. Why should developers not care about the non-functional requirements of their application when writing code? The answer in my opinion is not that developers should not care about it, it's that they don't want to care about it in most cases. Developers want to focus on innovations and pushing boundaries, not be shackled by the low level details of the cloud. This has been true through the history of software development, that we build abstraction layers on-top of implementation details. Most developers don't want to understand the inner workings of file systems and how they differ between operating systems. We just want to be able to read and write files, thus we have file system abstractions. Then if we need to handle special cases based on operating systems, or CPU architectures we expect the abstraction to give us a way to do that, without rewriting our entire code base.
Though I think it's worth noting that the purpose of the abstraction is not to hide implementation details and make cloud application development more vague, but rather unlock new mental models that drive innovation.
"Being abstract is something profoundly different from being vague … The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise" - Edsger Dijkstra
This is the "why" for our plugin system in Wing, it's intended to be a mechanism that supports teams to be more precise in their cloud applications. If the SDK unlocks this semantic level of thought, then the plugin system protects it.
The plugin system is a simple and powerful way to customize the compilation output of your Wing
application. It's comprised of a series of hooks that are called at various stages of the
compilation process. In our initial release of the plugin system we have made available 3 hooks:
preSynth, postSynth and validate. There are additional hooks that are currently in the think
tank, but we will save those for another blog post and focus on what we have available today.
To write a plugin all you need is to implement a JavaScript file, in which you can export one or all
of the compiler hooks. Once your plugin is written, you can use the --plugin flag in the wing cli
to include it in the compilation process.
wing compile -t tf-aws my-app.w --plugin my-plugin.js
The preSynth hook is executed after the construct tree has been initialized but before the code
has been synthesized to produce deployment artifacts. In which our plugins have the opportunity to
add and mutate resources in the construct tree.
The postSynth hook's execution is right after synthesis has been completed, and provides a means
in which we can manipulate the deployment artifacts (Terraform Config, CloudFormation template,
etc).
The validate hook is only executed after all compilation and synthesis has been completed. This is
important as this hook is meant to serve as a way to examine and validate deployment artifacts
without concerns that some later process will mutate them.
No blog post on a new feature would be complete without a walk-through :) So lets walk through the process of writing our own plugin. Not just any plugin though, but one that will help our favorite underappreciated platform engineer, Susan. As more teams have started adopting Wing in her organization she has realized that she can make use of plugins to help teams meet requirements for deploying applications into the cloud, without asking them to rewrite their code.
She has identified a common use case where her organization has deployed an IAM role into every AWS Account using nested stacks. The existence of this role as a permission boundary for all IAM roles is enforced through AWS organization SCPs (Service Control Policies). Thus, without it no IAM role can be created in the account, this is a cause of friction for teams that want to get their Wing applications deployed quickly. (whew thats a whole lot of things a developer should not have to care about)
She has decided to write a plugin that implements 2 hooks preSynth and validate. During the
preSynth hook, she wants to add the required permission boundary to all IAM roles in the construct
tree. Then she intends to validate the existence of the permission boundary on all roles during the
validate hook. This way teams can know their app will fail to deploy at compile time rather than
deploy, making for faster feedback loops.
Susan starts with writing the bare necessities of a plugin. She creates a file named
permission-boundary-compliance.js and adds the following code:
// Add permission boundary to all IAM roles
exports.preSynth = function (app) { }
// Validate that all IAM roles have a permission boundary
exports.validate = function (config) { }
She plans to make use of a concept from CDKTF known as Aspects to traverse the construct tree and add the permission boundary. She can safely use this since she knows her intended target will be Terraform for AWS.
const iam_role = require("@cdktf/provider-aws/lib/iam-role");
const cdktf = require("cdktf");
class PermissionBoundaryAspect {
constructor(permissionBoundaryArn) {
this.permissionBoundaryArn = permissionBoundaryArn;
}
visit(node) {
if (node instanceof iam_role.IamRole) {
node.permissionsBoundary = this.permissionBoundaryArn;
}
}
}
// Add permission boundary to all IAM roles
exports.preSynth = function (app) {
if (!process.env.PERMISSION_BOUNDARY_ARN) {throw new Error("env var PERMISSION_BOUNDARY_ARN not set")}
cdktf.Aspects.of(app).add(new PermissionBoundaryAspect(process.env.PERMISSION_BOUNDARY_ARN))
}
So above we can see she created a new Aspect class that implements the visit method. Each node the
aspect visits will be checked to determine if it is an IAM role and if so, set the permission
boundary which was passed into the plugin through an environment variable PERMISSION_BOUNDARY_ARN.
Finally for her validate step, she will simply traverse the Terraform config for all IAM roles and
check if the permission boundary is set. Even though her preSynth hook will have already done the
job, she knows that preSynth is a mutable hook and that another plugin may have altered things
after.
// Validate that all IAM roles have a permission boundary
exports.validate = function (config) {
for (const iamRole of Object.keys(config.resource.aws_iam_role)) {
const role = config.resource.aws_iam_role[iamRole];
if (!role.permission_boundary) {
throw new Error(`Role ${iamRole} does not have a permission boundary`);
}
if (role.permission_boundary !== process.env.PERMISSION_BOUNDARY_ARN) {
throw new Error(`Role ${iamRole} has incorrect permission boundary. Expected: ${process.env.PERMISSION_BOUNDARY_ARN} but got: ${role.permission_boundary}}`);
}
}
}
Now Susan can use the plugin in her CD pipelines to ensure that all IAM roles have the correct permission boundary set, without imposing this non-functional requirement on the application developers. Susan will go on to write more plugins to help her organization meet their security and compliance requirements. She is no longer the underappreciated platform engineer we know from the beginning of our blog post, but rather a hero with her own corner office, private parking spot, and an on call pager that never goes off.
The outcome of her success is purely speculative. Your company may not have corner offices so there is a chance you will have to just settle for the parking spot.
This new plugin system is very exciting, and has a lot of possibilities. However, if it is to ever reach its full potential we need your help! If you have some ideas for useful plugins, or thoughts on additional hooks, or even just questions about how to make use of the plugin system, we want to hear from you! Open a pull request or an issue on our GitHub also join our community slack and let us know what you think.
Want to read more about Wing plugins? Check out our plugin documentation for more information on the plugin system. For more code examples visit our plugin code examples
There are two ways to create resources in the cloud: in preflight, or in inflight. In this post, I'll explore what these terms mean, and why I think most cloud applications should avoid dynamically creating resources in inflight and instead stick to managing resources in preflight using tools like IaC.
Today, the cloud computing revolution has made it easier than ever to build applications that scale to meet the demands of users. However, as the cloud has become more prevalent, it has also become more complex.
One of the important questions you'll have to answer in order to build an application with AWS, Azure, or Google Cloud is: how should I create the cloud resources for my application?
For simple applications, you can get away with creating resources by clicking around in the cloud console. But as your application grows, a more structured approach is necessary. Infrastructure as code (IaC) tools like Terraform and CloudFormation have become popular for this purpose.
In general, there are two ways to create cloud resources for an application: before the application starts running, as part of the deployment process, and while the application is running, as part of the data path. We refer to these two phases of the application's lifecycle as preflight and inflight. Clever, ha?
In the cloud ecosystem, many cloud services do not make a hard distinction
between APIs that manage resources and APIs that use those resources. For
example, in AWS's documentation for SQS, operations like CreateQueue and
SendMessage are listed side by
side.
The same goes for Google Cloud's Pub/Sub
service.
However, there are significant differences between these two types of APIs in practice. This post will explore why I believe most cloud applications should avoid dynamically creating resources in inflight and, instead, focus on managing resources in preflight using tools like IaC.
First, dynamic resource creation introduces enormous complexity from a resource management perspective. This is the main reason why the IaC tools were created. Not only is it too cumbersome and error-prone to create large numbers of cloud resources by clicking buttons in your web browser, but it also becomes difficult to reliably maintain, update, and track the infrastructure. This is especially true as you start to pay attention to the cost of your application.
When you use tools like Terraform or CloudFormation, you typically create a YAML file or JSON file that describes resources in a declarative format. These solutions have several benefits:
When resources are created, updated, and deleted dynamically as part of an application's data path, we lose many of these benefits. I’ve heard of many cases where an application was designed around creating resources dynamically, and entire projects and teams had to be dedicated just to writing code that garbage collects these resources.
There are a few kinds of applications that require dynamic resource creation of course (like applications that provision cloud resources on behalf of other users), but these tend to be the exception to the rule.
Second, dynamic resource creation can make your application more likely to encounter runtime errors in production. Resource creation and deletion typically requires performing control plane operations on the underlying cloud provider, while most inflight operations only require data plane operations.
Cloud services are more fault tolerant when they only depend on data plane operations as part of the business logic's critical path. This is because even if the control plane of a cloud service has a partial outage (for example, if AWS Lambda functions could not be updated with new code), the data plane can continue running with the last known configuration, even as servers come in and out of service. This property, called static stability, is a desirable attribute in distributed systems, and most cloud platforms are designed around these tradeoffs.
Lastly, dynamic resource creation means your code needs to have admin-like permissions, which dramatically increases the attack surface for bad actors.
In the cloud, most machines ultimately need some form of network access - whether it’s to connect with other VMs in a cluster, or to connect to other cloud services (like automatically scaling databases and messaging queues).
When resources are statically defined, you can narrowly scope these permissions to define which resources are exposed to the public, which resources can call which endpoints, and even which teams can view sensitive data (and how data accesses are logged and audited).
I believe the best way to write applications for the cloud is to define your resources in preflight, and then use them in inflight. That's why Wing, the programming language my team and I are building, encourages developers to create resources in preflight as the easiest path to follow. We think the distinction between preflight and inflight is critical, which is why we've built it into the language itself. For example, if you try to create a resource in a block of code that is labeled with an inflight scope, Wing will produce a compiler error:
bring cloud;
let queue = new cloud.Queue();
queue.on_message(inflight (message: str) => {
// error: Cannot create the resource "Bucket" in inflight phase.
new cloud.Bucket();
});
Wing is intended to be a general purpose language, so you'll still be able to make API calls to a cloud providers (through network requests or JavaScript/TypeScript libraries) to dynamically create resources if you really want to. But in these scenarios, Wing won't provide resource management capabilities or generate resource permissions for you, so it would be your responsibility to manage the resource and ensure they get cleaned up.
If you're curious to learn more, check out our getting started guide or join us on our community slack and share what kinds of applications you're building in the cloud! We would love to hear your feedback about this design -- and if you have use case where dynamically creating resources would be helpful, please share it with us through a GitHub issue or on this blog's discussion post! ❤️
Chris Rybicki has recently added support for let var to Wing (see
the pull request), and I thought it might be a good
opportunity to share our thoughts on the topic of immutability in Wing.
One of Wing's design goals is to help developers write safer code. Change in state is a major source of complexity (and bugs) in software. Eric Elliott's Dao of Immutability describes it beautifully:
"The true constant is change. Mutation hides change. Hidden change manifests chaos. Therefore, the wise embrace history"
A language-level guarantee that state cannot change offers opportunities for caching, runtime optimizations and lock-free concurrency. Those attributes are very useful in distributed systems.
This is why, similarly to other modern programming languages such as Rust and Go, we are designing Wing to be immutable by default.
Let's look at an example:
let my_array = [1,2,3,4];
The above code defines an immutable array with the contents [1,2,3,4] and assigns it to
my_array. Immutability means that the contents of the object cannot be modified.
So if we try to add an item:
my_array.push(5);
// ^^^^ Unknown symbol "push"
Eventually we would want this error to be something like
Operation "push" is only available on mutable arrays. Did you mean to declare the array with MutArray<num>?, but bear with us...
This is because the type of my_array is Array<num>, which represents an immutable array, it
simply doesn't have any methods that will cause it to change. In Wing, the following types are
immutable: str, num, bool, Array<T>, Set<T> and Map<T>.
If I wanted to define it as a mutable array, I will need to be explicit:
let my_mut_array = MutArray<str>["hello", "world"];
And now we can go wild:
my_mut_array.push("go wild!"); // OK!
Similarly, we can define other mutable collection types:
let my_set = MutSet<str>{"hello", "world"};
let my_map = MutMap<bool>{"dog": true, "cat": false};
By the way: we are still debating if the standard types should be pascal-cased (e.g.
Array<T>,MutArray<T>) or snake (array<T>,mut_array<T>). Let us know what you think!
Yes! We are going to make this slightly harder to define mutable collections.
In the future, maybe we will introduce some syntactic sugar like:
let x = mut [1,2,3]; // <-- not a doctor
This design concept is what's called "good cognitive friction" (or "mechanical sympathy"). It is introduced intentionally in order to make sure the user understands the system better and encourage best practices.
But immutability is not enough! Since we reference our array through my_array, the compiler also
needs to guarantee that my_array will always point to the same object.
Let's look at a hypothetic example:
let i = 10;
new cloud.Function(inflight () => { print(i); }) as "f1";
i = 20;
new cloud.Function(inflight () => { i = i + 9; }) as "f2";
i = i - 90;
What value will the cloud function print? We can't tell because i is reassigned in multiple
locations and there is absolutely no way to determine its value.
This is where reassignability comes into play. In fact, in Wing, the above example would have failed compilation:
i = 20;
// ^ variable i is not reassignable
OK, now we can relax. The Wing compiler tells us that i is not reassignable.
Reassignability is a form of mutability (it is mutating the reference) and most modern programming
languages are trying to encourage single assignment. let in Rust, := in Go, and const
everywhere in JavaScript.
So how do you make something reassignable? You can use let var:
let var s = "hello";
s = "world";
You can also use var in class and resource declarations:
class Foo {
i: num;
var s: str;
init() {
// all non-optional fields must be assigned at construction (not implemented yet)
this.i = 10;
this.s = "world";
}
bar() {
// "var" fields can be reassigned at any time
this.s = "hello";
this.i = 20;
// ^ i is not reassignable
}
}
It can also be used in argument declarations:
let handler = inflight (var x: str) => {
if x == "hello" {
x = "${x} world";
}
};
let var?We originally considered using var instead of let
var, but we realized this is making it too easy to do the wrong thing. Entire code bases will be
written with just var and mountains of linters will be written to protect you from shooting
yourself in the foot.
Going back to this concept of "good cognitive friction". If you need to type a few more characters in
order to make a variable reassignable (let var versus let), you will likely just use let most
of the time, and the world will be a better place with less bugs and happier developers.
So how is all this related to cloud development?
One of the very cool things about immutable state is that the compiler can create as many copies of it as needed. If the compiler has a guarantee that a blob of data will never change over the lifetime (and space) of the system, it can simply distribute it where it is needed.
This means, that in Wing, immutable data can be seamlessly referenced from any inflight context.
Let's look at a very simple example just to explain the idea:
bring cloud;
let my_array = ["hello", "world"];
new cloud.Function(inflight (_: str) => {
assert(my_array.length == 2);
}) as "test";
So what's going on here? We have defined a cloud function that simply references my_array. As much
as this looks simple and intuitive, the compiler actually had to do a bit of work to make this
happen. As a reminder, a cloud.Function represents a cloud compute platform (such as AWS Lambda).
This means that the code inside the inflight block is going to be executed sometime in the future,
on some other machine. Completely isolated from the original memory space in which my_array was
defined.
Since our array is immutable, the compiler can safely clone it and bundle it together with the code that runs inside the cloud function.
In the future, the compiler will be able to identify that
my_array.lengthitself is immutable, and will only copy its value (see #1251).
If we try to reference a reassignable variable from inflight code:
let var s = "hello";
new cloud.Function(inflight (_: str) => {
print(s);
// ^ Cannot capture a reassignable variable "s"
});
If we try to reference a mutable collection from inflight code:
bring cloud;
let my_array = MutArray<num>[1,2,3,4];
new cloud.Function(inflight (_: str) => {
assert(my_array.length == 4);
// ^^^^^^^^ Cannot reference 'my_array' of type 'MutArray<num>' from an inflight context
});
In this case as well, the compiler won't allow us to reference a mutable object within an inflight context, because it won't be able to guarantee correctness.
Unsupported yet, but we will also have clone() to cover you in case you want to reference a
snapshot of a mutable collection (clone_mut() returns a mutable clone):
let mut_arr = MutArray<num>[1,2,3];
let arr = mut_arr.clone();
new cloud.Function(inflight () => {
assert(arr.length == 3);
});
See this pull request if you are curious how immutable capturing works in Wing (for the time being).
In the current revision of the language specification, we still haven't covered the idea of immutable user-defined types (its on our roadmap).
This means that the compiler only allows capturing primitives, Array, Map, Set, Json (coming
soon) and structs (coming soon). Any other type cannot be captured directly. This means you will
likely need to extract any information from the object in order to reference it within an inflight
context.
There are endless ways to express ideas using code and we believe a programming language should be
designed to make it intuitive for developers to write better, safer and more robust code. We use
"good cognitive friction" such as let var and MutXxx to get our brain to spare another cognitive
cycle on choosing some programming approach.
Making Wing "immutable by default" is designed to encourage developers to write more functional and immutable code. We continue to think of how to do it in elegant, simple, and not annoying ways, and we would love your feedback and suggestions on Wing Slack.
Welcome to the first ever issue of the Wing Inflight Magazine, our periodic update on what's up (yes, two puns) with the Wing Programming Language, and the community that is starting to gather around the mission to democratize cloud development.
A manifesto for cloud-oriented programming.
Don't get me wrong, I love the cloud! It has empowered me to build amazing things, and completely changed the way I use software to innovate and solve problems.
It's the "new computer", the ultimate computer, the "computerless computer". It can elastically scale, it's always up, it exists everywhere, it can do anything. It's boundless. It's definitely here to stay.
But holy crap, there is no way this is how we are going to be building applications for the cloud in the next decade. As the cloud evolved from "I don't want servers under my desk" to "my app needs 30 different managed services to perform its tasks", we kind of lost track of what a great developer experience looks like.