Notes
Search…
How To?

Manipulate File

Check if a file exist

1
if [ ! -f /tmp/foo.txt ]; then
2
echo "File not found!"
3
fi
Copied!

Find all files with specific extension of a directory and its subdirectories

1
find . -type f -name "*.conf"
Copied!

Parse file with shell script

1
#!/bin/sh
2
set -e
3
4
cmd="/usr/local/bin/docker-entrypoint.sh"
5
configFile="/usr/config/memcached.conf"
6
7
# parseConfig parse the config file and convert it to argument to pass to memcached binary
8
parseConfig() {
9
args=""
10
while read -r line || [ -n "$line" ]; do
11
case $line in
12
-*) # for config format -c 500 or --conn-limit=500
13
args="$(echo "${args}" "${line}")"
14
;;
15
[a-z]*) # for config format conn-limit = 500
16
trimmedLine="$(echo "${line}" | tr -d '[:space:]')" # trim all spaces from the line (i.e conn-limit=500)
17
param="$(echo "--${trimmedLine}")" # append -- in front of trimmedLine (i.e --conn-limit=500)
18
args="$(echo "${args}" "${param}")"
19
;;
20
\#*) # line start with #
21
# commented line, ignore it
22
;;
23
*) # invalid format
24
echo "\"$line\" is invalid configuration parameter format"
25
echo "Use any of the following format\n-c 300\nor\n--conn-limit=300\nor\nconn-limit = 300"
26
exit 1
27
;;
28
esac
29
done <"$configFile"
30
cmd="$(echo "${cmd}" "${args}")"
31
}
32
33
# if configFile exist then parse it.
34
if [ -f "${configFile}" ]; then
35
parseConfig
36
fi
37
# Now run docker-entrypoint.sh and send the parsed configs as arguments to it
38
$cmd
Copied!

Create lots of random files

1
seq -w 1 10 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=file.% bs=$(shuf -i1-10 -n1) count=1024'
Copied!

Manipulate String

Check first character of a string

Using wildcard

1
str="/some/directory/file"
2
if [[ $str == /* ]]; then
3
echo 1;
4
else
5
echo 0;
6
fi
Copied!

Using substring expansion

1
if [[ ${str:0:1} == "/" ]] ; then
2
echo 1;
3
else
4
echo 0;
5
fi
Copied!
This is going to take a substring of str starting at the 0th character with length 1.

Using Regex

1
if [[ $str =~ ^/ ]];then
2
echo 1;
3
else
4
echo 0;
5
fi
Copied!
^ indicates starting with.

Remove space from a string

1
str=" This sentence contains leading trailing and intermediate whitespaces "
2
3
## Remove leading whitespaces
4
lstr="$( echo "${str}" | sed -e 's/^[[:space:]]*//')"
5
echo "$lstr"
6
7
## Remove trailing whitespaces
8
tstr="$( echo "${str}" | sed -e 's/[[:space:]]*$//')"
9
echo "$tstr"
10
11
## Remove all white spaces
12
astr="$( echo "${str}" | tr -d '[:space:]')"
13
echo "$astr"
Copied!

Check if a string/line exist in a file

1
if grep -Fxq "$WhatToFind" my_list.txt
2
then
3
# code if found
4
else
5
# code if not found
6
fi
Copied!
Here,
  • F: Affects how PATTERN is interpreted (fixed string instead of a regex)
  • x: Match whole line
  • q: Shhhhh... minimal printing
Example:
1
if grep -q ^"$prefix:" out.yaml; then
2
echo "$prefix found in out.yaml"
3
else
4
echo "Not found"
5
fi
Copied!
This check if out.yaml file has a line starting with $prefix.

Replace whole line in of a file that match a pattern

1
sed -i "/string or regex goes here/c\replace with this" fileName
2
Copied!
Here, -i for replace in place and c is for change/replace.
Example:
1
sed -i "/^$key:/c\\$line" out.yaml
Copied!
This replace the line that start with $key of out.yaml file with $line.

Use Bearer Token

With wget

1
wget --header="Authorization: Bearer ${TOKEN}" ${URL}
Copied!
Example:
1
wget --header="Authorization: Bearer ${TOKEN}" https://sandbox.xill.io/v2/contents/5a184d0207903113023b5aaa/SANDBOX.md --no-check-certificate
Copied!

With curl

1
curl -H 'Accept: application/json' -H "Authorization: Bearer ${TOKEN}" ${URL}
Copied!
Example:
1
curl -H 'Accept: application/json' -H "Authorization: Bearer ${TOKEN}" https://{hostname}/api/myresource
Copied!

Work with Executable

Check an executable file static build or dynamic build

Use this command to check if a file is statistically linked or dynamically linked.
1
$ ldd ./yq
2
not a dynamic executable
Copied!
or
1
$ file yq
2
yq: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, with debug_info, not stripped
Copied!
Last modified 1yr ago