Upload Csv File Along With Lambda Function + Serveless Framework

Apply Serverless to create a Residuum API with Node.js and Lambda

Lately, I've been turning to AWS Lambda for edifice server-side logic — whether for client work, product evolution, or even personal projects. Lambda is a managed service, then there'south no need to patch or monitor servers. Information technology's pay-as-yous-go, so you simply become charged for usage, rather than uptime. And it's elastic, so it scales up to handle enterprise level traffic, or shrinks to zero for those pet projects that never take off.

In that location's ordinarily a lot more to an app than simply the Lambda role. API Gateway gives your Lambda a consumer-facing REST endpoint. And IAM policies grant your Lambda access to other AWS services. As your Lambda integrates with more AWS services, managing your app settings can become complicated and mistake-decumbent.

The Serverless framework simplifies the process of building and maintaining Lambda applications. Here are a few highlights:

• Infrastructure as code: AWS settings are divers in a configuration file that can be versioned in source control.
• Single point of alter: App settings for Lambda, API Gateway, and IAM are consolidated into a single configuration file.
• Unproblematic deployment: Serverless packages and deploys your Lambda app to AWS with a single command.

In that location are other tools out there to assist you manage your Lambda applications. For example, Chalice from AWS Labs supports Lambdas written in Python. This tutorial focuses on Serverless.

Annotation: The term "serverless architecture" refers to a way of building applications that run without having to manage infrastructure (like an e'er-on Linux box). The "Serverless framework" is a 3rd party tool that helps you manage and deploy your app to AWS Lambda and API Gateway.

Using Serverless, you'll create a Node.js Residual API that responds with a JSON assortment describing the contents of an S3 bucket. Your app architecture will end up looking similar this:

What yous need for this tutorial

All of the steps are performed on a Mac, then yous may need to adapt them if you're using Windows or Linux.

You'll need an AWS account. If you don't have one, you can sign up for the free tier.

If don't have Homebrew, install it with the following control:

          /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"        

Using Homebrew, install Node.js and Python.

brew install node mash install python

Afterwards yous install Python, install the AWS CLI using pip:

sudo pip install awscli --ignore-installed vi

one. Install Serverless using NPM

The Serverless framework is an NPM module. To install it, type the following command into Terminal:

$ npm install -one thousand serverless

The -1000 flag installs Serverless globally, which gives you the convenience of running the serverless  command from whatsoever directory.

Pro tip: You can install the NPM module local to your projection, instead of globally.

$ npm install serverless --save-dev

The Serverless team likes to motion fast and break things, so it might be a good idea to set your Serverless version in your package.json. This style, you lot can utilize the latest version of Serverless on new projects without impacting older ones.

The tradeoff is that you'll have to drill down to a subfolder in guild to reach the serverless executable:

$ ./node_modules/serverless/bin/serverless

two. Create a sample project

Serverless has commands, like create , deploy , and invoke . You're going to start off with using the create control.

In an empty folder, blazon the following:

$ serverless create --template aws-nodejs

This creates a new serverless project using the congenital-in Node.js template.

Note: There's too a template called aws-python  for you python developers out there.

The create command generates 2 new files — one for code, and the other for configuration:

handler.js:  This file contains your Lambda code.

'utilize strict'; module.exports.hello = (effect, context, callback) => { // 1   const response = { // 2     statusCode: 200,     body: JSON.stringify({       message: 'Go Serverless v1.0! Your role executed successfully!',       input: event, // iii     }),   };   callback(null, response); // 4 };        

There's a single function chosen module.exports.hello  that responds with a JSON object. Hither are a few things worth pointing out:

1. The result object contains data almost how the Lambda was triggered.
2. The response  object is a JSON payload with some boilerplate information that yous will eventually return to the caller.
3. The event object is reflected dorsum to the caller for debugging purposes.
4. Yous execute the callback  when you're ready to respond.

serverless.yml:  This is the Serverless configuration file.

service: aws-nodejs provider:   name: aws   runtime: nodejs4.3  # 1 functions:   hello:  # 2     handler: handler.hello  # iii        

Note: The default YAML file has a lot of comments and whitespace, but you can come across a cleaner version using this command:

$ sed '/^[#;].*$/d;/^$/d' serverless.yml

Information technology'due south in YAML format, which is like JSON but uses indentation instead of curly braces. Y'all'll exist seeing a lot of the serverless.yml file throughout the tutorial. Just for now, just pay attention to these lines:

1. runtime : Y'all're using the Node.js runtime. Here's a list of other runtimes that Lambda supports.
2. hello : This is your function name.
3. handler : This is the file and path to your function.

Give your Lambda part a try by using the invoke  control.:

$ sls invoke local -f howdy

Even though it's simply a unmarried line, at that place are a couple things going on here:

• sls :This is just a shortcut for serverless .
• local:Your function runs on your local machine. Don't worry, it'south not touching AWS at this point.
• -f hullo: You use the office flag and specify hello  as an argument.

You should get a response:

{   "statusCode": 200,   "body": "{\"bulletin\":\"Go Serverless v1.0! Your function executed successfully!\",\"input\":\"\"}" }        

So far, you created a Serverless project using a Node.js sample template. Then you invoked the hello function on your local machine.

Now that everything is working locally, information technology'south time to deploy to the cloud.

3. Create a service account in AWS

Serverless needs access keys in order to perform actions within your AWS account. And so you need to create a service business relationship with a set of access keys.

Perform the following steps inside the AWS console:

1. Get to IAM
2. On the left, select theUserstab
3. Click theAdd user button
4. For the User name, typeserverless
5. Next to Access type, check the box for Programmatic access
6. Click theNext: Permissions push button

Using the Add user wizard, yous begin the process of creating a service account named serverless. You too selectProgrammatic access, which generates access keys for you.

Continue with the next department of the Add together User sorcerer.

1. On the Set permissions for serverless page, clickAttach existing policies directly
2. Type AdministratorAccess in the search filter
3. Check the box next to theAdministratorAccesspolicy
4. Click theAdjacent: Reviewbutton
5. On the Review page, click theCreate userbutton
6. Click theDownload .csv push button
7. ClickShut

You grant admin privileges to your service account by attaching theAdministratorAccess policy. Once the account is created, you download a CSV file containing the access keys. This is the merely chance yous become to download these keys.

Notation:In a production environment, you lot should tailor down access to least privilege. This tutorial uses a blanket admin access policy to keep things simple. Be sure to detach this policy when you're done.

four. Configure a local AWS profile

Now that you have a gear up of access keys, you can salvage them inside an AWS profile on your local Mac. Later, you will refer to this contour name in the Serverless configuration file.

Note: It's a good exercise to apply AWS profiles so you don't accidentally deploy infrastructure to the incorrect AWS account.

Create a local AWS profile named serverless:

$ aws configure --profile serverless

You lot will exist prompted with a serial of questions. Refer to the downloaded CSV file when filling out the Access and Secret Access keys:

AWS Access Key ID [None]: ABCEDFGHIJKLMNOPQRST AWS Hugger-mugger Access Key [None]: 1122334455aaBBccDDeeFFggHHiiJJkk//LLmmOO Default region name [None]: u.s.-eastward-1 Default output format [None]: json

Exam out your AWS profile with this control:

$ aws s3 ls --profile serverless

You lot should see a list of S3 buckets in your AWS account.

5. Deploy Serverless to AWS

At present that you have a local AWS contour, you can deploy your Serverless app to AWS.

Update serverless.yml with the following:

service: serverless-tutorial  # 1 provider:   proper name: aws   runtime: nodejs4.3   profile: serverless  # ii functions:   hullo:     handler: handler.how-do-you-do

You are making 2 changes:

1. Yous rename the service toserverless-tutorial. A service is similar a project.
2. You specify the local AWS profile you created in the previous section.

Deploy the app to AWS with the following command:

$ sls deploy

You should meet the following output:

Serverless: Creating Stack... Serverless: Checking Stack create progress... .....  Serverless: Stack create finished...  Serverless: Packaging service...  Serverless: Uploading CloudFormation file to S3...  Serverless: Uploading service .zip file to S3 (583 B)...  Serverless: Updating Stack...  Serverless: Checking Stack update progress...  ...............  Serverless: Stack update finished...  Service Data  service: serverless-tutorial  stage: dev  region: the states-east-ane  api keys:    None  endpoints:    None  functions:    serverless-tutorial-dev-howdy: arn:aws:lambda:united states of america-due east-1:123456789012:role:serverless-tutorial-dev-how-do-you-do        

To make sure that everything is working, invoke your lambda function from the command line:

$ sls invoke -f hello

You should come across the following response:

{   "statusCode": 200,   "body": "{\"message\":\"Go Serverless v1.0! Your function executed successfully!\",\"input\":{}}" }

Congratulations! You but deployed your first Serverless app to AWS.

Behind the scenes, Serverless is actually doing a lot of scaffolding. When you lot deployed the app, you may have noticed the following output:

• Uploading CloudFormation file...
• Uploading service .zip file to S3...

Serverless is using CloudFormation to manage multiple services like Lambda, S3, IAM, and more. Hither'due south a detailed diagram of how they work together:

• Lambda: Serverless creates a Lambda and deploys your code.
• S3: Your Lambda code is stored in an S3 bucket.
• CloudWatch: Lambda logs events to CloudWatch where you can view errors and panel statements.
• IAM role: In society for your Lambda to write events to CloudWatch, Serverless creates an IAM role policy that grants this permission.
• CloudFormation: Everything is created using CloudFormation. This means you can update your configuration, and roll changes back cleanly.

6. Create an HTTP endpoint

Upwards until this indicate, you lot've been invoking your Lambda from the command line. In this department, yous're going to use API Gateway to create a client-facing REST API layer.

The Serverless framework makes it piece of cake to add a new endpoint. Only add a few lines to your serverless.yml  file:

service: serverless-tutorial provider:   proper name: aws   runtime: nodejs4.three   profile: serverless functions:   hello:     handler: handler.hello     events:  # 1       - http:  # 2           method: go  # 3           path: howdy  # iv

This creates a Become HTTP endpoint using the relative path of / hello :

• events : Lambda functions tin can receive events from other services. API Gateway and S3 are mutual ways to trigger Lambda functions.
• - http : This indicates that you're using API Gateway. (The hyphen represents an array item in YAML.)
• method : You're using the GET HTTP verb.
• path : You ascertain a relative path of / hello . This gets appended to an automobile-generated URL, which you lot'll encounter later.

Note: Be careful with indentation when working with arrays in YAML.

Deploy your updated configuration to AWS:

$ sls deploy

The output now shows a URL endpoint:

... endpoints: GET - https://abcd1234ef.execute-api.u.s.-east-ane.amazonaws.com/dev/hello

Paste this URL into a browser, and yous should meet the following:

{   "bulletin":"Go Serverless v1.0! Your function executed successfully!",   "input":{     "resources":"/hello",     "path":"/hello",     "httpMethod":"Become",     ...     "trunk":null,     "isBase64Encoded":false   } }

With API Gateway in front of your Lambda function, your architecture now looks like this:

7. Install the aws-sdk for Node

You can utilise Lambda equally a springboard to access other AWS services. The aws-sdk for Node.js is a pop NPM module that provides JavaScript objects for AWS services like S3.

To become started, type the following command:

$ npm init

When prompted, fill up out the post-obit fields. Since this is just a tutorial, merely accept the defaults by hitting Enter  ten times or and then.

name: (serverless-tutorial) version: (1.0.0) description: entry point: (handler.js) examination command: git repository: keywords: author: license: (ISC)

It should generate a package.json  file with the post-obit contents:

{   "name": "serverless-tutorial",   "version": "i.0.0",   "description": "",   "main": "handler.js",   "scripts": {     "examination": "echo \"Error: no exam specified\" && leave 1"   },   "author": "",   "license": "ISC" }

The package.json file keeps track of your node modules, dependencies, and versions.

At present enter the following control:

$ npm install aws-sdk --save

This installs the AWS SDK module. The --save flag keeps track of the module and version number in package.json.

Now you lot can access the AWS SDK from your node application. Update handler.js  with the following code:

'apply strict'; var AWS = require('aws-sdk');  // <-- this is the only line that inverse module.exports.hello = (event, context, callback) => {   const response = {     statusCode: 200,     body: JSON.stringify({       message: 'Become Serverless v1.0! Your role executed successfully!',       input: issue,      }),   };   callback(null, response); };

So far, y'all're but getting a handle to the aws-sdk  using require .

Deploy the updated code to make sure no issues pop up.

$ sls deploy

viii. List a bucket on S3

Now that the AWS SDK is installed, you tin start making SDK calls. In this section, you lot're going to list objects on S3.

Showtime, y'all need to create a saucepan on S3 that contains a file.

1. Inside the AWS console, go to S3 and click Create Bucket.
2. For the Saucepan name, pick something like serverless-tutorial-thorntech-12345. To avert a name collision, make sure you apply a unique bucket name.
3. For the Region, select US Standard, or whatever is closest to you lot.
4. Click Create
5. Click on the new bucket name
6. Under Actions, clickUpload.
7. Click Add Files to upload a small test file of your option.

When y'all upload a file, just employ any text file or image. It doesn't actually matter what you use, equally long as information technology'southward modest.

To listing your saucepan contents, update handler.js  with the post-obit code:

'use strict'; var AWS = require('aws-sdk'); var s3 = new AWS.S3();  // 1 module.exports.hello = (outcome, context, callback) => {   var params = {     Bucket: 'serverless-tutorial-thorntech-12345',  // 2   };   s3.listObjectsV2(params, office(err, data) {  // three     if (err) {       console.log(err, err.stack);     } else {       const response = {         statusCode: 200,         body: JSON.stringify({           "bucket_list": information // 4         }),       };       callback(goose egg, response);     }   }); };

Here'southward what's going on:

1. AWS.S3():The AWS module has child objects for each AWS service. Here, you become a handle to the i for the S3 service.
2. params:You're going to use a method that expects an object with a key proper noun of Saucepan . Apply the name of the bucket you lot created before.
3. s3.listObjectsV2:You pass your object parameter into the listObjectsV2 method. The second parameter is a callback, which gives you admission to bucket list data.
4. data: Y'all include the bucket listing in the Lambda response.

Deploy the updated code to AWS.

$ sls deploy

Visit the url, and you should come across an Internal Server Mistake.

{"bulletin": "Internal server error"}

This error message doesn't actually requite you lot much data. To run into what's really going on, you'll need to use CloudWatch:

1. Become to CloudWatch in the AWS console
2. On the left, click on the Logs tab
3. Click on the Log Group for/aws/lambda/serverless-tutorial-dev-howdy.
4. Click on the latest Log Stream.
5. Look for whatsoever error letters.

Yous should see an mistake message that says Access Denied. The problem is that your Lambda does not take permission to read from your S3 bucket. In the side by side section, yous'll fix this using IAM.

9. Grant access using IAM

Behind the scenes, Serverless generates an IAM role policy for y'all. Just the just affair you become out of the box is the ability to write logs to CloudWatch. For anything across this, yous demand to explicitly grant access.

Update serverless.yml  with the following:

service: serverless-tutorial provider:   name: aws   runtime: nodejs4.3   profile: serverless   iamRoleStatements:  # 1     - Effect: Let  # 2       Action: S3:ListBucket  # iii       Resources: arn:aws:s3:::serverless-tutorial-thorntech-12345  # 4 functions:   hello:     handler: handler.hullo     events:       - http:           method: go           path: hello

You grant your Lambda the ability to list the contents of an S3 bucket:

1. iamRoleStatements: This accepts an array of IAM statements.
2. Effect: You can specify either Permit or Deny.
3. Action: This is a permission that lets you list the contents of a bucket. Refer to the documentation for more available actions.
4. Resource: This is the S3 bucket yous are trying to list. Remember to replace the bucket name with the i you created earlier.

IAM policies are in JSON format, and they look something like this. Serverless lets you specify IAM permissions directly within the YAML file, so you don't accept to mess with JSON.

Deploy to AWS once again.

$ sls deploy

Visit the url, and this time you should see a listing of the bucket contents:

{   "bucket_list": {     "Contents": [{       "Key":"testfile.txt",       "Size": 12     }],     "Name":"serverless-tutorial-thorntech-12345",   } }

For my saucepan, I accept a unmarried file called testfile.txt  that is 12 bytes.

Your app infrastructure now looks like this:

Your Lambda is accessing data from an S3 bucket. And using the Serverless configuration file, y'all grant your Lambda IAM permissions to listing bucket contents.

Conclusion

Using the Serverless framework, you created an API in Node.js that lists the contents of an S3 saucepan. And you configured multiple AWS services right from the Serverless configuration file:

• Lambda: You specified the runtime and handler.
• API Gateway: You created a GET HTTP endpoint.
• IAM: You granted your Lambda access to S3.

Hopefully this tutorial gives you a running get-go with the Serverless framework. To learn more, cheque out the documentation.

Got any tips for using Serverless? Have any suggestions for future tutorial topics? Feel costless to add your thoughts to the comments.

Like this mail service? Please share it using the share buttons to the left. And then join our mailing listing below and follow united states of america on Twitter – @thorntech – for futurity updates.

noehishave.blogspot.com

Source: https://www.thorntech.com/aws-tutorial-intro-using-lambda-serverless-framework/