merge 2 files line by line and delete duplicate words - shell script , awk

The following shell script main aim is to merge two files line by line and delete the repeated word or duplicate word from each line.

file 1 :
apple goa orange
hello hi how are you
file 2 :
mango goa orange apple
how are you hi hello
Final out put required is :
apple goa orange mango

hello hi how are you
Screen outputs :
nc041031:/opt/krish # cat 1.txt
apple goa orange
hello hi how are you
nc041031:/opt/krish # cat 2.txt
mango goa orange apple
how are you hi hello
nc041031:/opt/krish # cat merge.sh
#!/bin/sh

awk 'NR==FNR{a[FNR]=$0;next} {print a[FNR],$0}' 1.txt 2.txt | tee a.txt
awk '{ while(++i<=NF) printf (!a[$i]++) ? $i FS : ""; i=split("",a); print "" }' a.txt | tee final.txt

nc041031:/opt/krish # ./merge.sh
apple goa orange mango goa orange apple
hello hi how are you  how are you hi hello
apple goa orange mango
hello hi how are you
nc041031:/opt/krish # cat final.txt
apple goa orange mango
hello hi how are you


2 comments to "merge 2 files line by line and delete duplicate words - shell script , awk"

Post a comment

Whoever writes Inappropriate/Vulgar comments to context, generally want to be anonymous …So I hope U r not the one like that?
For lazy logs, u can at least use Name/URL option which doesn’t even require any sign-in, The good thing is that it can accept your lovely nick name also and the URL is not mandatory too.
Thanks for your patience
~Krishna(I love "Transparency")

Popular Posts

Enter your email address:

Buffs ...

Tags


Powered by WidgetsForFree