2017/12/20

F*%$@ systemd

Just today I installed latest Ubuntu on my machine.
I thought it would be wise to check if there are any unnecessary services running which listen on public interface. And guess what?

#netstat -anutp
tcp        0      0 0.0.0.0:5355            0.0.0.0:*               LISTEN      1015/systemd-resolv

What the heck is that?
Do I realy need a local DNS server exposed to the world?

Quick reserch:
https://www.cvedetails.com/cve/CVE-2017-9217/
https://www.theregister.co.uk/2017/06/29/systemd_pwned_by_dns_query/

Good God ... how not to hate systemd?

Kill it with fire!!!
# systemctl stop systemd-resolved
# systemctl disable systemd-resolved
# systemctl mask systemd-resolved
# echo "nameserver your-gateway-router-ip" > /etc/resolv.conf



2017/08/02

Debian: How to mount qcow2 disk image on host



If you ever need a way of modifying content of qcow2 disk images, here is how to do it:

sudo apt-get install qemu-utils
sudo modprobe nbd max_part=8


sudo qemu-nbd --connect=/dev/nbd0 /path/to/qcow2/image

The first command loads the nbd kernel module. The "max_part=N" option specifies the maximum number of partitions we want to manage with nbd.

Check the partitions:
sudo fdisk /dev/nbd0 -l

Finally, choose any one partition (e.g., /dev/nbd0p1) and mount it to a local mount point (e.g., /mnt).
sudo mount /dev/nbd0p1 /mnt

2017/06/06

Windows 10 Limited Connectivity fix



Recently, my sister had problem with laptop runing windows 10. It lost the internet contectivity after recent "Wana cry fixes". Surely the fix was not directly connected with the issue (it has to be some other patch or coincidence).

TLDR; To fix the issue try following commands:

netsh winsock reset catalog (Reset WINSOCK entries to installation defaults)
netsh int ipv4 reset reset.log (Reset IPv4 TCP/IP stack to installation defaults)

I had only limited knowledge what really was the cause since I didn't had access to the computer. The only way to interact was giving instruction to my sister to type proper commands ;)

So first let's try to ping the gateway.

C:\Users\Leszek>ping 192.168.1.1

Pinging 192.168.1.1 with 32 bytes of data:
Reply from 192.168.1.50: Destination host unreachable.
Reply from 192.168.1.50: Destination host unreachable.
Reply from 192.168.1.50: Destination host unreachable.
Reply from 192.168.1.50: Destination host unreachable.

Ping statistics for 192.168.1.1:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),


Suspecting that host does not have IPv4 assigned by DHCP, lets check:

C:\Users\Leszek>ipconfig /all

Windows IP Configuration

Host Name . . . . . . . . . . . . : DESKTOP-UOL3AHR
Primary Dns Suffix . . . . . . . :
Node Type . . . . . . . . . . . . : Hybrid
IP Routing Enabled. . . . . . . . : No
WINS Proxy Enabled. . . . . . . . : No
DNS Suffix Search List. . . . . . : home

[SNIP] ...

Wireless LAN adapter Wi-Fi 2:


Connection-specific DNS Suffix . : home
Description . . . . . . . . . . . : Atheros AR9271 Wireless Network Adapter
Physical Address. . . . . . . . . : 74-EA-3A-90-27-44
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes
IPv6 Address. . . . . . . . . . . : 2a01:11af:2d9:cb00:b9a0:fb37:6b91:4be4(Preferred)
Temporary IPv6 Address. . . . . . : 2a01:11af:2d9:cb00:b49e:1d17:8dc1:b53d(Preferred)
Link-local IPv6 Address . . . . . : fe80::b9a0:fb37:6b91:4be4%7(Preferred)
Autoconfiguration IPv4 Address. . : 169.254.75.228(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.0.0
Default Gateway . . . . . . . . . : fe80::8616:f9ff:fef9:2ef6%7
DHCPv6 IAID . . . . . . . . . . . : 477424186
DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-1E-18-3A-61-00-1D-09-40-2D-77
DNS Servers . . . . . . . . . . . : 2a01:11af:2d9:cb00:8616:f9ff:fef9:2ef6
NetBIOS over Tcpip. . . . . . . . : Enabled
Connection-specific DNS Suffix Search List :
home
[SNIP] ...

Yup, seems that autoconfiguration kicks in:
Autoconfiguration IPv4 Address. . : 169.254.75.228(Preferred)

But ... if so ... than what is this 192.168.1.50 which reply for ping. Usually this is local ip, but ... local IP is different. So ... it seems to be remote IP and the routing table is broken?
Lets try to examine both tables:

C:\Users\Leszek>route print
===========================================================================
Interface List
29...72 ea 3a 90 27 44 ......Microsoft Hosted Network Virtual Adapter
19...00 1d 09 40 2d 77 ......Marvell Yukon 88E8040 PCI-E Fast Ethernet Controller
15...00 ff ac 02 bb f6 ......Kaspersky Security Data Escort Adapter
7...74 ea 3a 90 27 44 ......Atheros AR9271 Wireless Network Adapter
1...........................Software Loopback Interface 1
8...00 00 00 00 00 00 00 e0 Microsoft Teredo Tunneling Adapter
18...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter
===========================================================================

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination Netmask Gateway Interface Metric
127.0.0.0 255.0.0.0 On-link 127.0.0.1 331
127.0.0.1 255.255.255.255 On-link 127.0.0.1 331
127.255.255.255 255.255.255.255 On-link 127.0.0.1 331
169.254.0.0 255.255.0.0 On-link 169.254.75.228 306
169.254.75.228 255.255.255.255 On-link 169.254.75.228 306
169.254.255.255 255.255.255.255 On-link 169.254.75.228 306
224.0.0.0 240.0.0.0 On-link 127.0.0.1 331
224.0.0.0 240.0.0.0 On-link 169.254.75.228 306
255.255.255.255 255.255.255.255 On-link 127.0.0.1 331
255.255.255.255 255.255.255.255 On-link 169.254.75.228 306
===========================================================================
Persistent Routes:
Network Address Netmask Gateway Address Metric
0.0.0.0 0.0.0.0 192.168.1.1 Default
===========================================================================

IPv6 Route Table
===========================================================================
Active Routes:
If Metric Network Destination Gateway


C:\Users\Leszek>arp -a -v

Interface: 127.0.0.1 --- 0x1
Internet Address Physical Address Type
224.0.0.22 static
224.0.0.252 static
239.255.255.250 static

Interface: 169.254.75.228 --- 0x7
Internet Address Physical Address Type
169.254.255.255 ff-ff-ff-ff-ff-ff static
224.0.0.22 01-00-5e-00-00-16 static
224.0.0.251 01-00-5e-00-00-fb static
224.0.0.252 01-00-5e-00-00-fc static
239.255.255.250 01-00-5e-7f-ff-fa static

Interface: 0.0.0.0 --- 0xffffffff
Internet Address Physical Address Type
224.0.0.22 01-00-5e-00-00-16 static
224.0.0.252 01-00-5e-00-00-fc static
239.255.255.250 01-00-5e-7f-ff-fa static

Interface: 0.0.0.0 --- 0xffffffff
Internet Address Physical Address Type
169.254.75.228 00-00-00-00-00-00 invalid
169.254.153.184 00-00-00-00-00-00 invalid
192.168.1.1 84-16-f9-f9-2e-f6 invalid
192.168.1.2 00-00-00-00-00-00 invalid
192.168.1.4 00-00-00-00-00-00 invalid
192.168.1.15 00-00-00-00-00-00 invalid
192.168.1.255 ff-ff-ff-ff-ff-ff static
224.0.0.22 01-00-5e-00-00-16 static
224.0.0.251 01-00-5e-00-00-fb static
224.0.0.252 01-00-5e-00-00-fc static
224.0.0.253 01-00-5e-00-00-fd static
239.255.255.250 01-00-5e-7f-ff-fa static

Interface: 0.0.0.0 --- 0xffffffff
Internet Address Physical Address Type
224.0.0.22 01-00-5e-00-00-16 static
224.0.0.252 01-00-5e-00-00-fc static
239.255.255.250 01-00-5e-7f-ff-fa static


There is no 192.168.1.50 either in ARP not routing table ... so what is this IP? Is this local IP somewhere in crappy windows IP stack or some cached IP? What is going on?
How the heck Windows administrator are debugging such things?

My Windows are limited (god bless) and I didn't want to force my sister to run Wireshark to solve the mystery about this 192.168.1.50. For sure no such host exist in the network, and TTL suggest that it is some magic IP used by local windows stack.

Fortunately my brother found following "magic" commands xD, which reset the windows IP stack. xD
You get it ... no clue what is going on, so let's reset all the magic inside windows stack to default xD

C:\WINDOWS\system32>netsh winsock reset catalog

Sucessfully reset the Winsock Catalog.
You must restart the computer in order to complete the reset.

C:\WINDOWS\system32>netsh int ipv4 reset reset.log
Resetting Global, OK!
Resetting Interface, OK!
Resetting Unicast Address, OK!
Resetting Neighbor, OK!
Resetting Path, OK!
Resetting Route, OK!
Resetting , failed.
Odmowa dostępu.

Resetting , OK!
Restart the computer to complete this action.



And it works after reset ...
God ... I HATE Windows!

Edit:
After applying fix we ping this strange host, to confirm it wasn't present on network:

C:\Users\Leszek>Ping 192.168.1.50
Pinging 192.168.1.50 with 32 bytes of data:
Reply from 192.168.1.5: Destination host unreachable.
Reply from 192.168.1.1: Destination host unreachable.
Reply from 192.168.1.1: Destination host unreachable.
Reply from 192.168.1.1: Destination host unreachable.
Ping statistics for 192.168.1.50:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),


How Windows admins are able to debug things on this crappy operating system is still mystery for me.

Update: some other sources adds following commands:

netsh int reset restlog.txt
netsh int TCP set heuristics disabled
netsh int TCP set global autotuninglevel=disabled
netsh int tcp set global RSS=enabled

2017/03/05

Why 4k is the optimal sector size for modern USB flash drives

Everyone has this special day in a year when you receive gifts (they call it birthday but I'm too old already to celebrate. Old enough to try to forgot that earth make yet another full round around the sun and your head become more "blond" than you checked last time ;)

So I got SanDisk 64GB USB flash drive which is advertised to give 100MB/s of transfer speed. It uses USB 3.0 so I decided to give it a try and verify those claims. Another reason was my brother (who gave me this) "taunts" that "Windows handle it better" ;)



So I made simple "dd" test, with RAW read and write without any file system.

 # sh -c "sync && echo 3 > /proc/sys/vm/drop_caches"  
 # dd of=/dev/sdc if=/dev/zero bs=64M oflag=dsync,direct iflag=fullblock & echo $!; while sudo kill -SIGUSR1 $!; do sleep 1; done  
 [1] 22786  
 22786  
 0+0 records in  
 0+0 records out  
 0 bytes copied, 0,00882205 s, 0,0 kB/s  
 1+0 records in  
 1+0 records out  
 67108864 bytes (67 MB, 64 MiB) copied, 1,82117 s, 36,8 MB/s  
 2+0 records in  
 2+0 records out  
 134217728 bytes (134 MB, 128 MiB) copied, 3,39514 s, 39,5 MB/s  
 3+0 records in  
 3+0 records out  
 201326592 bytes (201 MB, 192 MiB) copied, 5,00308 s, 40,2 MB/s  
 4+0 records in  
 4+0 records out  
 268435456 bytes (268 MB, 256 MiB) copied, 6,58802 s, 40,7 MB/s  
 5+0 records in  
 5+0 records out  
 335544320 bytes (336 MB, 320 MiB) copied, 8,37488 s, 40,1 MB/s  
 # killall dd  

 # sh -c "sync && echo 3 > /proc/sys/vm/drop_caches"  
 # dd if=/dev/sdc of=/dev/null bs=64M iflag=fullblock,dsync,direct & echo $!; while sudo kill -  
 [2] 31336  
 31336  
 1+0 records in  
 0+0 records out  
 0 bytes copied, 0,57976 s, 0,0 kB/s  
 2+0 records in  
 1+0 records out  
 67108864 bytes (67 MB, 64 MiB) copied, 1,13625 s, 59,1 MB/s  
 4+0 records in  
 3+0 records out  
 201326592 bytes (201 MB, 192 MiB) copied, 2,25124 s, 89,4 MB/s  
 6+0 records in  
 5+0 records out  
 335544320 bytes (336 MB, 320 MiB) copied, 3,36255 s, 99,8 MB/s  
 8+0 records in  
 7+0 records out  
 469762048 bytes (470 MB, 448 MiB) copied, 4,47168 s, 105 MB/s  
 10+0 records in  
 9+0 records out  
 603979776 bytes (604 MB, 576 MiB) copied, 5,58709 s, 108 MB/s  
 11+0 records in  
 10+0 records out  
 671088640 bytes (671 MB, 640 MiB) copied, 6,14454 s, 109 MB/s  
 13+0 records in  
 12+0 records out  
 805306368 bytes (805 MB, 768 MiB) copied, 7,25453 s, 111 MB/s  
 15+0 records in  
 14+0 records out  
 939524096 bytes (940 MB, 896 MiB) copied, 8,3586 s, 112 MB/s  
 17+0 records in  
 16+0 records out  
 1073741824 bytes (1,1 GB, 1,0 GiB) copied, 9,46021 s, 114 MB/s  

Looks like ~40MB/s for write and ~100MB/s for read. So .... yes, this is as advertised by SanDisk. You just need to look at those little stars placed near the numbers. You will find their meaning at the other side of the box. They claim 100MB/s for reading and up to 10x times of standard drive speed (4MB/s) for writing.

But I wonder how well my system behave when I would use this drive together with some file system. The most obvious choice is VFAT.
mkfs is your best friend. Let's use default parameters then.


 # mkfs.vfat /dev/sdc1  
 mkfs.fat 3.0.28 (2015-05-16)  
 # fsck.vfat /dev/sdc1 -v  
 fsck.fat 3.0.28 (2015-05-16)  
 Checking we can access the last sector of the filesystem  
 Boot sector contents:  
 System ID "mkfs.fat"  
 Media byte 0xf8 (hard disk)  
 512 bytes per logical sector  
 32768 bytes per cluster  
 32 reserved sectors  
 First FAT starts at byte 16384 (sector 32)  
 2 FATs, 32 bit entries  
 7580160 bytes per FAT (= 14805 sectors)  
 Root directory start at cluster 2 (arbitrary size)  
 Data area starts at byte 15176704 (sector 29642)  
 1894928 data clusters (62093000704 bytes)  
 32 sectors/track, 64 heads  
 2048 hidden sectors  
 121305055 sectors total  
 Checking for unused clusters.  
 Checking free cluster summary.  
 /dev/sdc1: 0 files, 1/1894928 clusters   
 # cd /media/rad/837C-93FB/  
 # dd of=./test1.txt if=/dev/zero bs=32M oflag=dsync mount=1000 & echo $!; while sudo kill -SIGUSR1 $!; do sleep 1; done  
 [1] 9897  
 9897  
 0+0 records in  
 0+0 records out  
 0 bytes copied, 0,0124351 s, 0,0 kB/s  
 0+1 records in  
 0+1 records out  
 18821120 bytes (19 MB, 18 MiB) copied, 1,0552 s, 17,8 MB/s  
 1+1 records in  
 1+1 records out  
 52375552 bytes (52 MB, 50 MiB) copied, 2,47131 s, 21,2 MB/s  
 2+1 records in  
 2+1 records out  
 85929984 bytes (86 MB, 82 MiB) copied, 4,14265 s, 20,7 MB/s  
 3+1 records in  
 3+1 records out  
 119484416 bytes (119 MB, 114 MiB) copied, 5,55161 s, 21,5 MB/s  
 4+1 records in  
 4+1 records out  
 153038848 bytes (153 MB, 146 MiB) copied, 7,0338 s, 21,8 MB/s  
 5+1 records in  
 5+1 records out  
 186593280 bytes (187 MB, 178 MiB) copied, 8,69431 s, 21,5 MB/s  
 6+1 records in  
 6+1 records out  
 220147712 bytes (220 MB, 210 MiB) copied, 10,1471 s, 21,7 MB/s  

What the f... ? ~21MB/s !!!!
So after a while I realized that this is probably due default sector size. Lets try to reformat for 4k sector size:

 # mkfs.vfat /dev/sdc1 -S 4096 -s 1  
 mkfs.fat 3.0.28 (2015-05-16)  
 # fsck.vfat /dev/sdc1 -v  
 fsck.fat 3.0.28 (2015-05-16)  
 Checking we can access the last sector of the filesystem  
 Boot sector contents:  
 System ID "mkfs.fat"  
 Media byte 0xf8 (hard disk)  
 4096 bytes per logical sector  
 4096 bytes per cluster  
 32 reserved sectors  
 First FAT starts at byte 131072 (sector 32)  
 2 FATs, 32 bit entries  
 60534784 bytes per FAT (= 14779 sectors)  
 Root directory start at cluster 2 (arbitrary size)  
 Data area starts at byte 121200640 (sector 29590)  
 15133546 data clusters (61987004416 bytes)  
 32 sectors/track, 64 heads  
 2048 hidden sectors  
 15163136 sectors total  
 Checking for unused clusters.  
 Checking free cluster summary.  
 /dev/sdc1: 0 files, 1/15133546 clusters  
 root@rad-desktop:/media/rad# cd  
 0AC2-20C2/ dysk/  
 root@rad-desktop:/media/rad# cd 0AC2-20C2/  
 root@rad-desktop:/media/rad/0AC2-20C2# dd of=./test.txt if=/dev/zero bs=64M oflag=dsync,direct iflag=fullblock & echo $!; while sudo kill -SIGUSR1 $!; do sleep 1; done  
 [1] 21689  
 21689  
 0+0 records in  
 0+0 records out  
 0 bytes copied, 0,0138234 s, 0,0 kB/s  
 1+0 records in  
 1+0 records out  
 67108864 bytes (67 MB, 64 MiB) copied, 2,02883 s, 33,1 MB/s  
 1+0 records in  
 1+0 records out  
 67108864 bytes (67 MB, 64 MiB) copied, 2,03412 s, 33,0 MB/s  
 2+0 records in  
 2+0 records out  
 134217728 bytes (134 MB, 128 MiB) copied, 3,77909 s, 35,5 MB/s  
 3+0 records in  
 3+0 records out  
 201326592 bytes (201 MB, 192 MiB) copied, 5,97208 s, 33,7 MB/s  
 4+0 records in  
 4+0 records out  
 268435456 bytes (268 MB, 256 MiB) copied, 7,93711 s, 33,8 MB/s  
 5+0 records in  
 5+0 records out  
 335544320 bytes (336 MB, 320 MiB) copied, 9,74674 s, 34,4 MB/s  
 6+0 records in  
 6+0 records out  
 402653184 bytes (403 MB, 384 MiB) copied, 11,8841 s, 33,9 MB/s  
 7+0 records in  
 7+0 records out  
 469762048 bytes (470 MB, 448 MiB) copied, 13,8475 s, 33,9 MB/s  
 ^C  

35MB/s ... hmmm. So the reason for this is native sector size of the flash drive. It seems that the device advertise 512B as native sector size but it internally use 4k. Because of this each write operation need to be combined in the firmware of the drive which degrades the performance.
I was to lazy to check, what the mkfs.vfat tools would do if drive would report 4k sectors. I suppose that it should use bigger FAT sectors but that's only a wild guess (or rather a wish).

Now, you probably wonder... How many of wrongly formatted flash drives are in your possession? ;)