Categories
hive

How to create external table in hive

In this post let us learn how to create external table in hive

Types of table in hive

The following are the two types of tables in the hive.

  1. Internal table or Managed table
  2. External table

For getting familiar with internal tables, Please refer to the below link.

https://beginnersbug.com/how-to-create-a-table-in-hive/  

What is External table ?

For creating an external table, we need to use one keyword called external.

The remaining syntax will remain the same for both types.

How to create ?

Following is the syntax for creating an external table.

Fields terminated by  ‘,’ – need to specify the delimiter of the file to be loaded into the table.

hive> create external table temp_details (year string,temp int,place string) 
> row format delimited 
> fields terminated by ','; 
OK Time taken: 0.093 seconds

To get familiar with loading the table, Please refer to the following link.

https://beginnersbug.com/how-to-load-data-into-a-hive-table/

Categories
hive

How to load data into a hive table

In this post , we will learn how to load data into a hive table .

Types of table in hive

The following are the two types of tables in the hive .

  1. Internal table or Managed table
  2. External table

In this post, let us discuss the internal tables and their loading ways.

What are the ways to load data into a table ?

There are three ways to load data into a hive table.

  • Loading data from a file 
  • Inserting values into a table 
  • Loading data from some other table 
1 . Loading data from a file

We can refer to the path of the file as below.  More importantly, we need to specify the details of the file like delimiter while creating the table itself.

Please refer to the below link to understand it clearly.

https://beginnersbug.com/how-to-create-a-table-in-hive/

hive> load data local inpath '/tmp/temp.csv' into table truck;
Loading data to table vehicle_details.truck
OK
Time taken: 0.846 seconds
hive> select * from truck limit 5;
OK
1901    50      bangalore
1903    48      kolkata
1900    26      pune
1902    12      darjeling
1903    23      delhi
Time taken: 0.954 seconds, Fetched: 5 row(s)
2 . Inserting values into a table 

We shall insert values into the table manually like below.

We can even add multiple records into the table in a similar way.

hive> insert into table truck1 values ('2020',65,'Chennai'); 

Let us verify whether the inserted data looks good.

hive> select * from truck1 where year='2020';
OK
2020    65      Chennai
Time taken: 0.161 seconds, Fetched: 1 row(s)
3 . Loading data from some other table 

We can create one more table with the following command.

hive> create table if not exists truck1
    > (year string , temp int , place string)
    > row format delimited
    > fields terminated by ','
    > ;
OK
Time taken: 0.099 seconds

As below, we can select the data from the existing table and load it into the new table.

hive> insert into table truck1 select * from truck where temp < 50;

After loading, we could find the new table loaded with the selected data.

hive> select * from truck1 limit 5;
OK
1903    48      kolkata
1900    26      pune
1902    12      darjeling
1903    23      delhi
1902    25      delhi
Time taken: 0.145 seconds, Fetched: 5 row(s)
Categories
hive

How to create a table in hive

In this post, we will learn how to create a table in hive .

Creating a database in hive

In Hive, we either have to use an existing database or to create a new database before creating a table.

The following show command lists the number of available databases in the hive.

hive> show databases; 
OK 
default Time taken: 0.009 seconds, Fetched: 1 row(s)

The create database command helps us to create a new database as shown below.

hive> create database vehicle_details;
OK
Time taken: 0.208 seconds 
hive> show databases;
OK
default
vehicle_details 
Time taken: 0.011 seconds, Fetched: 2 row(s)
Creating table

The create syntax contains the following options.

if not exists(optional) – If any table exists already, it will ignore the statement.

other options – The specifications of the file to be loaded with the table. 

tblproperties – can skip the header in the file with this property.

hive> create table if not exists truck 
    > (year string , temp int , place string )
    > row format delimited
    > fields terminated by ','
    > stored as textfile
    > tblproperties ("skip.header.line.count"="1");
OK
Time taken: 0.278 seconds
hive> show tables; 
OK 
deliveries 
matches 
truck 
Time taken: 0.012 seconds, Fetched: 3 row(s)
How the file looks like
cat /tmp/temp.csv |head -6
year|temp|place
1900,45,chennai 
1901,50,bangalore 
1903,48,kolkata 
1900,26,pune 
1902,12,darjeling 
Reference

https://cwiki.apache.org/confluence/display/Hive/Tutorial#Tutorial-Creating,Showing,Altering,andDroppingTables

Categories
Angular

pass value between two components using the route in angular

In this tutorial, we will learn about pass value between two components using the route in angular

In many scenarios, we need to pass one or two parameters from one component to another component

We can pass a value in various ways, But in this tutorial, we go to pass value between components using the route

For this tutorial, We are going to pass student id from the student list on click

By default, we will define routing as like below in-app-routing.module.ts

{ path: 'student', component: StudentDetailsComponent }

We need to pass id along with the student route like /student/1from the student list page

We can receive those route param value in student details page using ActivatedRoute

import { ActivatedRoute } from '@angular/router';
Sending value from first component

Here we are using pass value while clicking a hyperlink. via router link, we are defining the student id value as like below

<a [routerLink]="['/student', studentId]">{{name}} </a>
Routing Syntax

We need to mention in app.routing.module.ts about the parameter as like below
{ path: 'student/:id', component: StudentDetailsComponent }

app.routing.module.ts
import { NgModule } from '@angular/core';
import { Routes, RouterModule } from '@angular/router';
import { StudentDetailsComponent } from './component/student-details/student-details.component';


const routes: Routes = [
  { path: 'student/:id', component: StudentDetailsComponent }
];

@NgModule({
  imports: [RouterModule.forRoot(routes)],
  exports: [RouterModule]
})
export class AppRoutingModule { }
Receiving value in second component

As mentioned above we are going to use activedRoute to retrieve the value

this.studentId = this.activeParms.snapshot.paramMap.get("id");
Student-details-component.ts
import { Component, OnInit } from '@angular/core';
import { ActivatedRoute } from '@angular/router';

@Component({

  selector: 'app-student-details',
  templateUrl: './student-details.component.html',
  styleUrls: ['./student-details.component.css']
})
export class StudentDetailsComponent implements OnInit {

  constructor(private activeParms: ActivatedRoute) { }
  studentId: string;
  ngOnInit(): void {
    this.studentId = this.activeParms.snapshot.paramMap.get("id");
  }

}
Conclusion

In this post, we looked about to pass value between two components using the route in angular

Related Articles

https://beginnersbug.com/ng-if-else-condition-in-angular-with-example/

Categories
Angular

ng if else condition in angular with example

In this post, we will learn about ng if else condition in angular with example

NgIf the directive will be used to display DOM based on a condition. It is very useful to show or hide a HTML div

Sample Code
<div ngIf="show">
    <p>This is can be show</p>
</div>
import { Component, OnInit } from '@angular/core';
import { StudentService } from '../../service/student.service';
import { Student } from '../../model/student.model';

@Component({

  selector: 'app-student-details',
  templateUrl: './student-details.component.html',
  styleUrls: ['./student-details.component.css']
})
export class StudentDetailsComponent implements OnInit {

  constructor() { }

  show: boolean;

  ngOnInit(): void {   
    this.show=true;
  }

}

In the above example, we declared a boolean variable called as show. We are changing that boolean value to true on ngOnInit method

In the html file we are using that show boolean variable to display the paragraph tag

if the show boolean is false then the html div will not display

ngIf else Example

We need to define the else block inside a ng-template block. below is the example for ng if else condition

<div ngIf="show; else notshowing">
    <p>This is can be show</p>
</div>

<ng-template #notshowing>
    <p>This will be shown on else statment</p>
</ng-template>
Conclusion

In the above post, we learned about ngIf else condition in angular with example

Related Articles

http post request from angular with example

Categories
Angular

http post request from angular with example

In this post, we will learn to make http post request from angular with example

To communicate with backend server we are using HttpClient from @angular/common/http

Syntax
this.http.post<return-type>(url: string, body: any, options?);

In this example we are going to save a student details. To do that we need to invoke a http post method with student details

{  
  "firstname": "string", 
  "lastname": "string"
  "department": "string",
}

Now I am creating a model class for above json request like below

export class Student {    
    firstname: string;    
    lastname: string;
    department: string;
}

Now we need to create service class to make http post request from angular
creating service layer command:  ng g s student

import { Injectable } from '@angular/core';

@Injectable({
  providedIn: 'root'
})
export class StudentService {

  constructor() { }
  
}

Now we can create saveStudent method to save the student details in back end

 /**
   * In this method we are calling http post method 
   * @param student 
   */
  saveStudentDetails(student: Student): Observable<any>{
    return this.http.post<any>('http://localhost:8080/api/student',student);
  }
Student Service layer
import { Injectable } from '@angular/core';
import { HttpClient } from '@angular/common/http';
import { Observable } from 'rxjs';
import { Student } from '../model/student.model';
@Injectable({
  providedIn: 'root'
})
export class StudentService {

  constructor(private http: HttpClient) { }


  /**
   * In this method we are calling http post method 
   * @param student 
   */
  saveStudentDetails(student: Student): Observable<any>{
    return this.http.post<any>('http://localhost:8080/api/student',student);
  }
  
}

Now we can call this service class method from component, For example I am creating a simple form and component as like below

import { Component, OnInit } from '@angular/core';
import { StudentService } from '../../service/student.service';
import { Student } from '../../model/student.model';

@Component({

  selector: 'app-student-details',
  templateUrl: './student-details.component.html',
  styleUrls: ['./student-details.component.css']
})
export class StudentDetailsComponent implements OnInit {

  constructor(private studentService: StudentService) { }

  student: Student=null;

  ngOnInit(): void {   

  }

  saveStudent(){
    //Creating model class
    this.student.firstname="Beginners";
    this.student.lastname="Bug";
    this.student.department="Blooggin";
    
    //Call service layer from component 
    this.studentService.saveStudentDetails(this.student).subscribe(data=>{
      console.log("Student Added successfully");
    
    },error=>{
      console.log("There was some error ",error);
    });

  }

}
<form (ngSubmit)="saveStudent()">
<label>FirstName:</label> <br/><input type="text" name="firstname" placeholder="Enter First name"><br/>
<label>LastName:</label> <br/><input type="text" name="lastname" placeholder="Enter Last name"><br/>
<label>Department:</label> <br/><input type="text" name="department" placeholder="Enter Department"><br/><br/>
<button class="button" type="submit">Add</button>

</form>
Conclusion

We learned to make http post request from angular with example

Related Articles

call http rest service from angular

Categories
Angular

call http rest service from angular

In this tutorial, we will learn to call http rest service from angular

To understand this tutorial we should know the angular basics like components, service

Before starting we should have a component & service knowledge, Here we are going to use appcomponent as component and going to create a service to call http service

Navigate to src/app from vs code terminal, Execute below command to create service

ng g s app-service

app-service.service.ts & app-service.service.spec.ts will get created in src/app folder

C:\Users\Admin\angular-demo\src\app>ng g s app-service
CREATE src/app/app-service.service.spec.ts (378 bytes)
CREATE src/app/app-service.service.ts (139 bytes)

C:\Users\Admin\angular-demo\src\app>

Now open the app-service.ts file

import { Injectable } from '@angular/core';

@Injectable({
  providedIn: 'root'
})
export class AppServiceService {

  constructor() { }
}
Http Endpoint details

For this tutorial, I have created a endpoint which return student details

http://localhost:8080/api/student
Output Json
[
  {
    "id": 1,
    "department": "ECE",
    "firstname": "rajesh",
    "lastname": "kumar"
  }
]

Create a student model class to map the HTTP Response

student.model.ts

export class Student {
    department: string;
    firstname: string;
    id: number;
    lastname: string;
}

create a method to invoke http call. We are using HttpClient from @angular/common/http

import { Injectable } from '@angular/core';
import { HttpClient } from '@angular/common/http';
import { Observable } from 'rxjs';
import { Student } from './model/student.model';


@Injectable({
  providedIn: 'root'
})
export class AppServiceService {

  constructor(private http: HttpClient) { }

  getStudentDetails(): Observable<Student[]>{
    return this.http.get<Student[]>('http://localhost:8080/api/student');

  }

}

Now call the getStudentDetails() method from app.component.ts on ngOnInit

  ngOnInit(): void {
    this.appService.getStudentDetails().subscribe(data => {
      this.studentDetails=data;
    });

Below is the complete app.component.ts

import { Component } from '@angular/core';
import { AppServiceService } from './app-service.service';
import { Student } from './model/student.model';

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css']
})
export class AppComponent {
  
  title = 'angular-demo';

  constructor(private appService: AppServiceService) { }

   studentDetails: Student[];

  ngOnInit(): void {
    this.appService.getStudentDetails().subscribe(data => {
      this.studentDetails=data;
    });

  }
}

You can map this StudentDetails to your html as like below

<div *ngIf="studentDetails">
    <table>
        <thead>
            <tr>
                <td>ID</td>
                <td>FirstName</td>
                <td>LastName</td>
                <td>Department</td>
            </tr>
        </thead>
        <tbody>
            <tr *ngFor="let response of studentDetails">
                <td>
                    {{response.id}}
                </td>
                <td>
                    {{response.department}}
                </td>
                <td>
                    {{response.firstname}}
                </td>
                <td>
                    {{response.lastname}}
                </td>
            </tr>

        </tbody>

    </table>

</div>
<router-outlet></router-outlet>
Conclusion

I this post we learned to call http rest service from angular

Github

https://github.com/rkumar9090/angular-demo/blob/master/src/app/app-service.service.ts

Categories
security spring-boot

spring-boot-security using http basic authentication

In this post, we will learn about spring-boot-security using http basic authentication

In this tutorial we will use spring security where we can enable security for our spring applications

Dependency
<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-security</artifactId>
</dependency>

Add above dependency in your pom.xml

In this example, we are creating two rest endpoints, which I named as hello and hi

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class HomeController {

	@GetMapping("hello")
	public String getHello() {
		return "Hello";
	}

	@GetMapping("hi")
	public String getHi() {
		return "Hi";
	}

}

Now we can create new class SecurityConfig.java to configure our security
we need to add @Configuration @EnableWebSecurityannotations in our SecurityConfig.java and also we need to extend WebSecurityConfigurerAdapter 

And also we need to override two configure methods in SecurityConfig.java

In the first configure method we need to configure user and role details, Here I added two users with two roles 


import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;
import org.springframework.security.crypto.factory.PasswordEncoderFactories;
import org.springframework.security.crypto.password.PasswordEncoder;

@Configuration
@EnableWebSecurity
public class SecurityConfig extends WebSecurityConfigurerAdapter {

	@Override
	protected void configure(AuthenticationManagerBuilder auth) throws Exception {
		PasswordEncoder encoder = PasswordEncoderFactories.createDelegatingPasswordEncoder();
		auth.inMemoryAuthentication()
		.withUser("admin").password(encoder.encode("password")).roles("ADMIN").and()
		.withUser("user").password(encoder.encode("pass")).roles("USER");
	}

	@Override
	protected void configure(HttpSecurity http) throws Exception {
		http
		.authorizeRequests().antMatchers("/hello").permitAll()
		.anyRequest().authenticated().and().httpBasic();
		
	}

}

In the above snippet, we allowed /hello endpoint and remaining endpoints are authenticated with httpBasic authentication

We are using PasswordEncoder to encode the password

Run the application as spring boot application

Testing

Conclusion

The above example is a simple spring security authentication using http basic. Here we added user details in-memory authentication

Github

https://github.com/rkumar9090/spring-security

Categories
pyspark

Subtracting dataframes in pyspark

In this post , let us learn about Subtracting dataframes in pyspark.

Creating dataframes in pyspark

We can create two dataframes using the below program for further use.

Sample program
from pyspark.sql import SparkSession
from pyspark import SparkContext
sc = SparkContext()
spark = SparkSession(sc)
from pyspark.sql import Row<br># Creating the first dataframe df
df=sc.parallelize([Row(name='Gokul',Class=10,level1=480,level2=380,level3=280,level4=520,grade='A'),Row(name='Usha',Class=12,level1=670,level2=720,level3=870,level4=920,grade='A'),Row(name='Rajesh',Class=12,level1=180,level2=560,level3=660,level4=850,grade='B')]).toDF()
print("Printing the dataframe df below")
df.show()<br># Creating the second dataframe df1
df1=sc.parallelize([Row(name='Usha',Class=12,level1=670,level2=720,level3=870,level4=920,grade='A'),Row(name='Kumar',Class=9,level1=320,level2=650,level3=760,level4=580,grade='C')]).toDF()
print("Printing the dataframe df1 below")
df1.show()
Output
Printing the dataframe df below
+-----+-----+------+------+------+------+------+
|Class|grade|level1|level2|level3|level4|  name|
+-----+-----+------+------+------+------+------+
|   10|    A|   480|   380|   280|   520| Gokul|
|   12|    A|   670|   720|   870|   920|  Usha|
|   12|    B|   180|   560|   660|   850|Rajesh|
+-----+-----+------+------+------+------+------+
Printing the dataframe df1 below
+-----+-----+------+------+------+------+-----+
|Class|grade|level1|level2|level3|level4| name|
+-----+-----+------+------+------+------+-----+
|   12|    A|   670|   720|   870|   920| Usha|
|    9|    C|   320|   650|   760|   580|Kumar|
+-----+-----+------+------+------+------+-----+
Subtracting dataframes

The keyword subtract helps us in subtracting dataframes in pyspark.

In the below program, the first dataframe is subtracted with the second dataframe.

#Subtracting dataframes in pyspark
df2=df.subtract(df1)
print("Printing the dataframe df2 below")
df2.show()
Printing the dataframe df2 below
+-----+-----+------+------+------+------+------+
|Class|grade|level1|level2|level3|level4|  name|
+-----+-----+------+------+------+------+------+
|   10|    A|   480|   380|   280|   520| Gokul|
|   12|    B|   180|   560|   660|   850|Rajesh|
+-----+-----+------+------+------+------+------+

We can subtract the dataframes based on few columns also.

#Subtracting dataframes based on few columns
df3=df.select('Class','grade','level1').subtract(df1.select('Class','grade','level1'))
print("Printing the dataframe df3 below ")
df3.show()
Printing the dataframe df3 below
+-----+-----+------+
|Class|grade|level1|
+-----+-----+------+
|   10|    A|   480|
|   12|    B|   180|
+-----+-----+------+
Reference

http://spark.apache.org/docs/latest/api/python/pyspark.html?highlight=subtract#pyspark.RDD.subtract

Categories
java

automatic mouse mover in java

In this post, we will do automatic mouse mover in java

We can move the cursor with random directions using below code

Example
import java.awt.Robot;

public class MouseMover {

	public static void main(String[] args) {
		try {
			int xCord = 10;
			int yCord = 20;
			while (true) {

				Robot robot = new Robot();
				robot.mouseMove(xCord++, yCord++);
				System.out.println("Moving mouse to " + xCord + " " + yCord);
				Thread.sleep(10000);
			}

		} catch (Exception e) {
			e.printStackTrace();
		}
	}
}
Github

https://github.com/rkumar9090/student-example/blob/master/src/main/java/com/beginnersbug/example/mouse/MouseMover.java