Categories
aws sqs

sqs consumer in spring boot

In this tutorial, we will learn about sqs consumer in spring boot

AWS offers a Queue data structure with SQS products to produce and consume messages from the queue

SQS stands for Simple Queue Service

Create a spring boot project using spring boot initializr
https://start.spring.io/

Here I am using java 8 with maven.

Prerequisites

  • Queue url
  • access key
  • secret key id
  • SQS region

Dependency

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-aws-messaging</artifactId>
   <version>2.2.4.RELEASE</version>
</dependency>

SQS Config

We need to configure our spring boot application for the SQS producer.
In this class, you have to provide access key id and secret key, and region
Create a new class SQSConfig.java and the below code


import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.auth.DefaultAWSCredentialsProviderChain;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.sqs.AmazonSQSAsync;
import com.amazonaws.services.sqs.AmazonSQSAsyncClientBuilder;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.aws.messaging.config.SimpleMessageListenerContainerFactory;
import org.springframework.cloud.aws.messaging.config.annotation.EnableSqs;
import org.springframework.cloud.aws.messaging.core.QueueMessagingTemplate;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.core.task.AsyncTaskExecutor;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;

@Configuration
@EnableSqs
public class SQSConfig {

    @Value("${cloud.aws.region:ap-south-1}")
    private String awsRegion;

    @Value("${cloud.aws.credentials.accessKeyId}")
    private String accessKey;

    @Value("${cloud.aws.credentials.secretKey}")
    private String secretKey;

    @Bean
    @Primary
    public AmazonSQSAsync amazonSQSAsync() {
        return AmazonSQSAsyncClientBuilder.standard()
                .withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials(accessKey, secretKey)))
                .withRegion(Regions.fromName(awsRegion))
                .build();
    }

    @Bean
    public AWSCredentialsProvider credentialsProvider() {
        return new DefaultAWSCredentialsProviderChain();
    }

    @Bean
    public QueueMessagingTemplate queueMessagingTemplate() {
        return new QueueMessagingTemplate(amazonSQSAsync());
    }


    @Bean
    public SimpleMessageListenerContainerFactory simpleMessageListenerContainerFactory(AmazonSQSAsync amazonSQSAsync) {
        SimpleMessageListenerContainerFactory factory = new SimpleMessageListenerContainerFactory();
        factory.setAmazonSqs(amazonSQSAsync);
        factory.setAutoStartup(true);
        factory.setMaxNumberOfMessages(10);
        factory.setTaskExecutor(createDefaultTaskExecutor());
        return factory;
    }

    protected AsyncTaskExecutor createDefaultTaskExecutor() {
        ThreadPoolTaskExecutor threadPoolTaskExecutor = new ThreadPoolTaskExecutor();
        threadPoolTaskExecutor.setThreadNamePrefix("SQSExecutor - ");
        threadPoolTaskExecutor.setCorePoolSize(100);
        threadPoolTaskExecutor.setMaxPoolSize(100);
        threadPoolTaskExecutor.setQueueCapacity(2);
        threadPoolTaskExecutor.afterPropertiesSet();
        return threadPoolTaskExecutor;
    }

}

Message Consumer

With the above configuration, we can start the consumer using  @SqsListener

SQSListener

import org.springframework.cloud.aws.messaging.listener.SqsMessageDeletionPolicy;
import org.springframework.cloud.aws.messaging.listener.annotation.SqsListener;
import org.springframework.stereotype.Component;

@Component
public class SQSListener {

    @SqsListener(value = "${cloud.queue.name}", deletionPolicy = SqsMessageDeletionPolicy.ON_SUCCESS)
    public void processMessage(String message) {
        try {
            System.out.println("RECEIVED MESSAGE --> " + message);
        } catch (Exception e) {
            throw new RuntimeException("Cannot process message from SQS", e);
        }
    }
}

application.properties

cloud.aws.credentials.accessKeyId=***
cloud.aws.credentials.secretKey=***
cloud.queue.name=weather

Output

RECEIVED MESSAGE --> {"message":"1"}

GitHub

https://github.com/rkumar9090/sqs-consumer

Related Articles

Categories
aws sqs

sqs producer in spring boot

In this tutorial, we will learn about sqs producer in spring boot

AWS offers a Queue data structure with SQS products to produce and consume messages from the queue

SQS stands for Simple Queue Service

Create a spring boot project using spring boot initializr
https://start.spring.io/

Here I am using java 8 with maven.

Prerequisites

  • Queue url
  • access key
  • secret key id
  • SQS region

Maven dependency

Add the below dependency in your pom.xml

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-aws-messaging</artifactId>
</dependency>

SQS Config

We need to configure our spring boot application for the SQS producer.
In this class, you have to provide access key id and secret key, and region
Create a new class SQSConfig.java and the below code

@Configuration
public class SQSConfig {
  

    @Value("${cloud.aws.credentials.accessKeyId}")
    private String accessKey;

    @Value("${cloud.aws.credentials.secretKey}")
    private String secretKey;

    @Bean
    public QueueMessagingTemplate queueMessagingTemplate() {
        return new QueueMessagingTemplate(amazonSQSAsync());
    }

    @Bean
    @Primary
    public AmazonSQSAsync amazonSQSAsync() {
        return AmazonSQSAsyncClientBuilder.standard().withRegion(Regions.AP_SOUTH_1)
                .withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials(accessKey, secretKey)))
                .build();
    }
}

Message Producer

With the above configuration, we can start to publish the message to SQS using convertAndSend Method

@Component
public class SQSProducer {

    @Autowired
    private QueueMessagingTemplate queueMessagingTemplate;

    @Value("${cloud.aws.credentials.end-point}")
    private String endpoint;

    public String sendMessage(Pojo message) {
        queueMessagingTemplate.convertAndSend(endpoint,message);
        return "Successfully sent message to SQS";
    }
}

Now you can create one controller and send the message from rest endpoint

@RestController
@RequestMapping("/api/data/")
public class ProducerController {

    @Autowired
    private SQSProducer publisher;

    @PostMapping
    public String sendMessage(@RequestBody Pojo message) {
        return publisher.sendMessage(message);
    }
}

application.properties

cloud.aws.credentials.accessKeyId=****
cloud.aws.credentials.secretKey=*****
cloud.aws.credentials.end-point=https://sqs.ap-south-1.amazonaws.com/978751824592/producer
cloud.aws.region.static.auto=false
cloud.aws.stack.auto=false

If you are facing network issue, add the below snipped in the spring boot main class

@SpringBootApplication(
		exclude = {
				org.springframework.cloud.aws.autoconfigure.context.ContextInstanceDataAutoConfiguration.class,
				org.springframework.cloud.aws.autoconfigure.context.ContextStackAutoConfiguration.class,
				org.springframework.cloud.aws.autoconfigure.context.ContextRegionProviderAutoConfiguration.class
		}
)

Now you can hit the URL from the postman and you can see the message in SQS AWS console

To Create an access key id and secret key refer to this
https://beginnersbug.com/create-access-key-in-aws-console/

GitHub URL

https://github.com/rkumar9090/sqs-producer

Categories
aws

Create access key in aws console

In this tutorial, we will learn about Create access key in awsconsole

Access key used to create for IAM user or root user. It will consist of two parts

  • access key ID 
  •  secret access key

We can use these keys to sign in to aws programmatically. Below are the steps to create the keys

Navigate to https://aws.amazon.com/, log in with your credentials with the sign in to console option in the top right corner

Once you logged in to the console, Search for IAM in the search box as shown below

Select the IAM option from the search result. You will redirect to the IAM management console

Select the My security credentials in Quick links as shown below

You will be redirect to another window which will have the security crendentials. Select the Access keys (access key ID and secret access key) in that list as shown below

Click on the Create New access key. It will generate the access key and secret key for you.

Copy the keys in the pop up. or download the keys.

Categories
data structures java two pointer

two pointer algorithm

Two pointer algorithm is one of the basic and easy data structures for beginners. It is also commonly asked in most of the interview

In this tutorial, we will learn to apply the two pointer algorithm to find the sum of two numbers in a sorted array

Input

given sorted array
let a = {8 , 12, 24, 30, 44, 54, 60, 61}
sum of two numbers 
let x = 98
Output
44,54

Algorithm explanation

  • The Array should be Sorted in ascending
  • initialize a variable as i with starting position as zero
  • initialize a variable as j with starting position as n-1
  • check the sum of index i & j, if matches then print the value
  • else if the sum is less than the target value then increase the i position
  • else decrease the j position

Example

public class TwoPointersExample {

    private void sumofTwoNumbers(int a[], int x) {
        if (a != null && a.length > 0) {
            int i = 0;
            int j = a.length - 1;
            boolean isFound = false;
            while (i < j) {
                int sum = a[i] + a[j];
                if (x == sum) {
                    System.out.println("The value is " + a[i] + " " + a[j]);
                    isFound = true;
                    break;
                } else if (sum < x) {
                    i++;
                } else {
                    j--;
                }
            }
            if (!isFound) {
                System.out.println("Not able to find with given input");
            }

        } else {
            System.out.println("Not a valid input");
        }

    }

    public static void main(String[] args) {
        int a[] = {8, 12, 24, 30, 44, 54, 60, 61};
        TwoPointersExample example = new TwoPointersExample();
        example.sumofTwoNumbers(a, 98);
        ;
    }
}

Output

The value is 44 54

Related Articles

Categories
Go

array in go lang

In this post, we will learn about array in go lang

In my previous tutorial, I have explained the installation of Go in windows. Please check out the below URL for that.
https://beginnersbug.com/install-go-lang-in-windows/

Arrays are basic in all programming languages which is used for data operations. We can have a collection of elements here.

Creating array in Go lang

Similar to other programming languages, we can create arrays with length and without length.

In the below example I created two arrays with indexes of 5 & 3 with the datatype of int and string

package main

// fmt package for user input & output
import "fmt"

// Main Method
func main() {
	// Creating an int array with index of 5
	var intarr = [5]int{10, 4, 6, 2, 12}
	// Creating an string array with index of 3
	var strarr = [3]string{"Iron Man", "Hulk", "Thor"}
	// Printing arrays
	fmt.Println(intarr)
	fmt.Println(strarr)
}

Output

[10 4 6 2 12]
[Iron Man Hulk Thor]

Printing index of array

In the below example, we will print the element based on the index value.
Here we are printing element at index intarr(1) & strarr(2)

package main

// fmt package for user input & output
import "fmt"

// Main Method
func main() {
	// Creating an int array with index of 5
	var intarr = [5]int{10, 4, 6, 2, 12}
	// Creating an string array with index of 3
	var strarr = [3]string{"Iron Man", "Hulk", "Thor"}
	// Printing arrays
	fmt.Println(intarr)
	fmt.Println(strarr)

	// Printing index of an array
	fmt.Println(intarr[1])
	fmt.Println(strarr[2])
}

Output

[10 4 6 2 12]
[Iron Man Hulk Thor]
4
Thor

The below snippet will show you to print the length of an array.
len() method is used to print the length of the array

package main

// fmt package for user input & output
import "fmt"

// Main Method
func main() {
	// Creating an int array with index of 5
	var intarr = [5]int{10, 4, 6, 2, 12}
	// Creating an string array with index of 3
	var strarr = [3]string{"Iron Man", "Hulk", "Thor"}
	// Printing arrays
	fmt.Println(intarr)
	fmt.Println(strarr)

	// Printing index of an array
	fmt.Println(intarr[1])
	fmt.Println(strarr[2])

	fmt.Println("Lenght of int array:", len(intarr))
	fmt.Println("Lenght of string array:", len(strarr))
}

Output

[10 4 6 2 12]
[Iron Man Hulk Thor]
4
Thor
Lenght of int array: 5
Lenght of string array: 3

Iterate array using for loop

Here we are printing each and every element with the help of for loop

package main

// fmt package for user input & output
import "fmt"

// Main Method
func main() {
	// Creating an int array with index of 5
	var intarr = [5]int{10, 4, 6, 2, 12}
	// Creating an string array with index of 3
	var strarr = [3]string{"Iron Man", "Hulk", "Thor"}
	// Printing arrays
	fmt.Println(intarr)
	fmt.Println(strarr)

	// Printing index of an array
	fmt.Println(intarr[1])
	fmt.Println(strarr[2])

	fmt.Println("Lenght of int array:", len(intarr))
	fmt.Println("Lenght of string array:", len(strarr))

	// Iterating array using for loop
	for i := 1; i < len(intarr); i++ {
		fmt.Print(intarr[i], " ")
	}
	fmt.Println()
	// Iterating array using for loop
	for i := 0; i < len(strarr); i++ {
		fmt.Print(strarr[i], " ")
	}

}

Output

[10 4 6 2 12]
[Iron Man Hulk Thor]
4
Thor
Lenght of int array: 5
Lenght of string array: 3
4 6 2 12 
Iron Man Hulk Thor

Go lang play

You can play with the array using their playgrounds also https://go.dev/tour/moretypes/6

Categories
Go

install go lang in windows

In this article, We will learn to install go lang in windows

GO lang is created by Google in 2007. It is the most prominent language now. It is easy to build fast and scale massive application

It is an open-source programming language supported by Google. Most of the cloud providers like google cloud, AWS, azure are supporting go lang in their ecosystem

Download MSI package

  1. Navigate to the this URL https://go.dev/doc/install 
  2. Click on the Download Go for Windows as like with below image 
  3. Now your download will get start automatically and ready to install
  4. Click on the downloaded .msi file
  5. Click on the next button on the installer prompt 
  6. Click the next for the consequtive steps.
  7. Once you installed you will get below screen in the installer prompt 

Verify the installation

Once you finished the installation you can verify the go lang via command prompt
Open the command prompt and run this command

go version

Categories
excel file python

read excel file in python

In this tutorial, we will learn about read excel file in python using the pandas library.

In this example, we are using the pandas library to read an excel file.

Pandas is an open-source library that has a lot of features inbuilt. Here we are using it for reading the excel file.
For More refer to official docs of the pandas
https://pandas.pydata.org/
For our exercise, I am going to use the below excel file format.

Pandas Installtion

pip install pandas

Read excel example

# Using pandas library
import pandas as panda


class ReadExcel:
    # Main method
    if __name__ == "__main__":
        # file path of xlsx file
        file_path = "D://data/Avengers.xlsx"
        # reading the excel file
        excel = panda.read_excel(file_path)
        print(excel)

Output

   ID   Character Name           Real Name
0   1             Hulk        Mark Ruffalo
1   2             Thor     Chris Hemsworth
2   3      Black Widow  Scarlett Johansson
3   4         Iron Man    Robert Downey Jr
4   5  Captain America         Chris Evans

References

https://beginnersbug.com/create-dataframe-in-python-using-pandas/

Related Articles

Categories
mongodb python

python with mongodb

In this post, we will see about connecting Python with MongoDB.

MongoDB is a fast and reliable NO SQL database that is used to store the data in JSON structure.

PyMongo

Here we are using the PyMongo library to connect MongoDB from python. refer https://pymongo.readthedocs.io/en/stable/

We already installed MongoDB, RoboMongo & pyhcarm for our development

I have created a database called avengers which have a collection as avengersdetails as like below image

PyMongo

Command to install pymongo library

pip install pymongo

PyMongo Import

import pymongo

In our example, we are using the OOPS concept to create the connection and fetch the data from Mongo DB.
Below line is the starting point of the code

if __name__ == "__main__":

Python with MongoDB Example

import pymongo


class MongoDbConn:

    # __init__ method will be used to initialize the variable
    # We need to pass the mongo db url & database name to this method
    def __init__(self, host_url, database):
        self.host_details = host_url
        self.db_name = database

    # Once you create the object you can connect the connection object from below method
    def get_mongodb_connection_details(self) -> object:
        my_client = pymongo.MongoClient(self.host_details)
        my_db = my_client[self.db_name]
        return my_db


if __name__ == "__main__":
    # Create an object for our class MongoDbConn
    obj = MongoDbConn("mongodb://localhost:27017/", "avengers")
    # Create database connection from the below line
    db_connection = obj.get_mongodb_connection_details()
    # pass your collection(table) name in below line
    collection_details = db_connection["avengersdetails"]
    for data in collection_details.find():
        print(data)

Output

{'_id': ObjectId('5fd0e603549a851a24a48c36'), 'ID': '1', 'Character Name': 'Hulk', 'Real Name': 'Mark Ruffalo'}
{'_id': ObjectId('5fd0e603549a851a24a48c37'), 'ID': '2', 'Character Name': 'Thor', 'Real Name': 'Chris Hemsworth'}
{'_id': ObjectId('5fd0e603549a851a24a48c38'), 'ID': '3', 'Character Name': 'Black Widow', 'Real Name': 'Scarlett Johansson'}
{'_id': ObjectId('5fd0e603549a851a24a48c39'), 'ID': '4', 'Character Name': 'Iron Man', 'Real Name': 'Robert Downey Jr'}
{'_id': ObjectId('5fd0e603549a851a24a48c3a'), 'ID': '5', 'Character Name': 'Captain America', 'Real Name': 'Chris Evans'}

Related Articles

Categories
data structures insertion sort java

insertion sort algorithm

In this article, we will discuss the simple sorting algorithm called insertion sort.

What is insertion sort?

This works in a similar fashion as playing a deck of cards. Assuming two different parts – sorted and unsorted, we need to pick and sort each card from an unsorted part into a sorted part.

Steps to be folowed

The first element in an array is to be considered as already sorted.

The next element is to be taken and compared with the elements in a sorted array.

Insert the element by shifting the elements in the sorted array which are greater than the value to be sorted

Example

import java.util.Arrays;

public class InsertionSort {

	private void sortArray(int[] arr) {

		for (int i = 1; i < arr.length; i++) {
			int key = arr[i];
			int j = i - 1;
			while (j >= 0 && arr[j] > key) {
				arr[j + 1] = arr[j];
				j--;
			}
			arr[j + 1] = key;
		}

	}

	public static void main(String[] args) {
		InsertionSort insertionSort = new InsertionSort();
		int[] arr = { 2, 7, 5, 8, 3, 4, 1, 6 };
		insertionSort.sortArray(arr);
		System.out.println(Arrays.toString(arr));
	}

}

Output

[1, 2, 3, 4, 5, 6, 7, 8]

Related Articles

Categories
python

for loop in python

In this tutorial, we will learn about for loop in python. We can iterate an array using for loop.

Syntax

for seq_val in sequence:
  body

Iterate array

cars = ["Honda", "Toyota", "Hyundai"]
for car in cars:
    print(car)

Output

Honda
Toyota
Hyundai

for loop using range

cars = ["Honda", "Toyota", "Hyundai"]
for i in range(len(cars)):
    print(cars[i])
Honda
Toyota
Hyundai

for loop with custom start value

cars = ["Honda", "Toyota", "Hyundai"]
for i in range(2, len(cars)):
    print(cars[i])

Output

Hyundai

Related Articles