So from time to time, it happens that I'm able to correctly predict the future.
According to this article, M$ just sold the last smartphones equipped with windows 10 mobile in their online store. The agony was a bit longer than I expected when I was predicting that "the fall of titans will be fast as thunder".
Anyway, it's a pity that Nokia ended like this.

F*%$@ systemd

Just today I installed latest Ubuntu on my machine.
I thought it would be wise to check if there are any unnecessary services running which listen on public interface. And guess what?

#netstat -anutp
tcp        0      0 0.0.0.0:5355            0.0.0.0:*               LISTEN      1015/systemd-resolv

What the heck is that?
Do I realy need a local DNS server exposed to the world?

Quick reserch:
https://www.cvedetails.com/cve/CVE-2017-9217/
https://www.theregister.co.uk/2017/06/29/systemd_pwned_by_dns_query/

Good God ... how not to hate systemd?

Kill it with fire!!!
# systemctl stop systemd-resolved
# systemctl disable systemd-resolved
# systemctl mask systemd-resolved
# echo "nameserver your-gateway-router-ip" > /etc/resolv.conf

Debian: How to mount qcow2 disk image on host

If you ever need a way of modifying content of qcow2 disk images, here is how to do it:

sudo apt-get install qemu-utils
sudo modprobe nbd max_part=8

sudo qemu-nbd --connect=/dev/nbd0 /path/to/qcow2/image

The first command loads the nbd kernel module. The "max_part=N" option specifies the maximum number of partitions we want to manage with nbd.

Check the partitions:
sudo fdisk /dev/nbd0 -l

Finally, choose any one partition (e.g., /dev/nbd0p1) and mount it to a local mount point (e.g., /mnt).
sudo mount /dev/nbd0p1 /mnt

Windows 10 Limited Connectivity fix

Recently, my sister had problem with laptop runing windows 10. It lost the internet contectivity after recent "Wana cry fixes". Surely the fix was not directly connected with the issue (it has to be some other patch or coincidence).

TLDR; To fix the issue try following commands:

netsh winsock reset catalog (Reset WINSOCK entries to installation defaults)
netsh int ipv4 reset reset.log (Reset IPv4 TCP/IP stack to installation defaults)
I had only limited knowledge what really was the cause since I didn't had access to the computer. The only way to interact was giving instruction to my sister to type proper commands ;)

So first let's try to ping the gateway.

C:\Users\Leszek>ping 192.168.1.1

Pinging 192.168.1.1 with 32 bytes of data:
Reply from 192.168.1.50: Destination host unreachable.
Reply from 192.168.1.50: Destination host unreachable.
Reply from 192.168.1.50: Destination host unreachable.
Reply from 192.168.1.50: Destination host unreachable.

Ping statistics for 192.168.1.1:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),

Suspecting that host does not have IPv4 assigned by DHCP, lets check:

C:\Users\Leszek>ipconfig /all

Windows IP Configuration

Host Name . . . . . . . . . . . . : DESKTOP-UOL3AHR
Primary Dns Suffix . . . . . . . :
Node Type . . . . . . . . . . . . : Hybrid
IP Routing Enabled. . . . . . . . : No
WINS Proxy Enabled. . . . . . . . : No
DNS Suffix Search List. . . . . . : home

[SNIP] ...

Wireless LAN adapter Wi-Fi 2:


Connection-specific DNS Suffix . : home
Description . . . . . . . . . . . : Atheros AR9271 Wireless Network Adapter
Physical Address. . . . . . . . . : 74-EA-3A-90-27-44
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes
IPv6 Address. . . . . . . . . . . : 2a01:11af:2d9:cb00:b9a0:fb37:6b91:4be4(Preferred)
Temporary IPv6 Address. . . . . . : 2a01:11af:2d9:cb00:b49e:1d17:8dc1:b53d(Preferred)
Link-local IPv6 Address . . . . . : fe80::b9a0:fb37:6b91:4be4%7(Preferred)
Autoconfiguration IPv4 Address. . : 169.254.75.228(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.0.0
Default Gateway . . . . . . . . . : fe80::8616:f9ff:fef9:2ef6%7
DHCPv6 IAID . . . . . . . . . . . : 477424186
DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-1E-18-3A-61-00-1D-09-40-2D-77
DNS Servers . . . . . . . . . . . : 2a01:11af:2d9:cb00:8616:f9ff:fef9:2ef6
NetBIOS over Tcpip. . . . . . . . : Enabled
Connection-specific DNS Suffix Search List :
home[SNIP] ...

Yup, seems that autoconfiguration kicks in:
Autoconfiguration IPv4 Address. . : 169.254.75.228(Preferred)

But ... if so ... than what is this 192.168.1.50 which reply for ping. Usually this is local ip, but ... local IP is different. So ... it seems to be remote IP and the routing table is broken?
Lets try to examine both tables:

C:\Users\Leszek>route print
===========================================================================
Interface List
29...72 ea 3a 90 27 44 ......Microsoft Hosted Network Virtual Adapter
19...00 1d 09 40 2d 77 ......Marvell Yukon 88E8040 PCI-E Fast Ethernet Controller
15...00 ff ac 02 bb f6 ......Kaspersky Security Data Escort Adapter
7...74 ea 3a 90 27 44 ......Atheros AR9271 Wireless Network Adapter
1...........................Software Loopback Interface 1
8...00 00 00 00 00 00 00 e0 Microsoft Teredo Tunneling Adapter
18...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter
===========================================================================

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination Netmask Gateway Interface Metric
127.0.0.0 255.0.0.0 On-link 127.0.0.1 331
127.0.0.1 255.255.255.255 On-link 127.0.0.1 331
127.255.255.255 255.255.255.255 On-link 127.0.0.1 331
169.254.0.0 255.255.0.0 On-link 169.254.75.228 306
169.254.75.228 255.255.255.255 On-link 169.254.75.228 306
169.254.255.255 255.255.255.255 On-link 169.254.75.228 306
224.0.0.0 240.0.0.0 On-link 127.0.0.1 331
224.0.0.0 240.0.0.0 On-link 169.254.75.228 306
255.255.255.255 255.255.255.255 On-link 127.0.0.1 331
255.255.255.255 255.255.255.255 On-link 169.254.75.228 306
===========================================================================
Persistent Routes:
Network Address Netmask Gateway Address Metric
0.0.0.0 0.0.0.0 192.168.1.1 Default
===========================================================================

IPv6 Route Table
===========================================================================
Active Routes:
If Metric Network Destination Gateway

C:\Users\Leszek>arp -a -v

Interface: 127.0.0.1 --- 0x1
Internet Address Physical Address Type
224.0.0.22 static
224.0.0.252 static
239.255.255.250 static

Interface: 169.254.75.228 --- 0x7
Internet Address Physical Address Type
169.254.255.255 ff-ff-ff-ff-ff-ff static
224.0.0.22 01-00-5e-00-00-16 static
224.0.0.251 01-00-5e-00-00-fb static
224.0.0.252 01-00-5e-00-00-fc static
239.255.255.250 01-00-5e-7f-ff-fa static

Interface: 0.0.0.0 --- 0xffffffff
Internet Address Physical Address Type
224.0.0.22 01-00-5e-00-00-16 static
224.0.0.252 01-00-5e-00-00-fc static
239.255.255.250 01-00-5e-7f-ff-fa static

Interface: 0.0.0.0 --- 0xffffffff
Internet Address Physical Address Type
169.254.75.228 00-00-00-00-00-00 invalid
169.254.153.184 00-00-00-00-00-00 invalid
192.168.1.1 84-16-f9-f9-2e-f6 invalid
192.168.1.2 00-00-00-00-00-00 invalid
192.168.1.4 00-00-00-00-00-00 invalid
192.168.1.15 00-00-00-00-00-00 invalid
192.168.1.255 ff-ff-ff-ff-ff-ff static
224.0.0.22 01-00-5e-00-00-16 static
224.0.0.251 01-00-5e-00-00-fb static
224.0.0.252 01-00-5e-00-00-fc static
224.0.0.253 01-00-5e-00-00-fd static
239.255.255.250 01-00-5e-7f-ff-fa static

Interface: 0.0.0.0 --- 0xffffffff
Internet Address Physical Address Type
224.0.0.22 01-00-5e-00-00-16 static
224.0.0.252 01-00-5e-00-00-fc static
239.255.255.250 01-00-5e-7f-ff-fa static

There is no 192.168.1.50 either in ARP not routing table ... so what is this IP? Is this local IP somewhere in crappy windows IP stack or some cached IP? What is going on?

How the heck Windows administrator are debugging such things?

My Windows are limited (god bless) and I didn't want to force my sister to run Wireshark to solve the mystery about this 192.168.1.50. For sure no such host exist in the network, and TTL suggest that it is some magic IP used by local windows stack.

Fortunately my brother found following "magic" commands xD, which reset the windows IP stack. xD
You get it ... no clue what is going on, so let's reset all the magic inside windows stack to default xD

C:\WINDOWS\system32>netsh winsock reset catalog

Sucessfully reset the Winsock Catalog.
You must restart the computer in order to complete the reset.

C:\WINDOWS\system32>netsh int ipv4 reset reset.log
Resetting Global, OK!
Resetting Interface, OK!
Resetting Unicast Address, OK!
Resetting Neighbor, OK!
Resetting Path, OK!
Resetting Route, OK!
Resetting , failed.
Odmowa dostępu.

Resetting , OK!
Restart the computer to complete this action.


And it works after reset ...
God ... I HATE Windows!

Edit:
After applying fix we ping this strange host, to confirm it wasn't present on network:

C:\Users\Leszek>Ping 192.168.1.50
Pinging 192.168.1.50 with 32 bytes of data:
Reply from 192.168.1.5: Destination host unreachable.
Reply from 192.168.1.1: Destination host unreachable.
Reply from 192.168.1.1: Destination host unreachable.
Reply from 192.168.1.1: Destination host unreachable.
Ping statistics for 192.168.1.50:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),

How Windows admins are able to debug things on this crappy operating system is still mystery for me.

Update: some other sources adds following commands:

netsh int reset restlog.txt
netsh int TCP set heuristics disabled
netsh int TCP set global autotuninglevel=disabled
netsh int tcp set global RSS=enabled

Why 4k is the optimal sector size for modern USB flash drives

Everyone has this special day in a year when you receive gifts (they call it birthday but I'm too old already to celebrate. Old enough to try to forgot that earth make yet another full round around the sun and your head become more "blond" than you checked last time ;)

So I got SanDisk 64GB USB flash drive which is advertised to give 100MB/s of transfer speed. It uses USB 3.0 so I decided to give it a try and verify those claims. Another reason was my brother (who gave me this) "taunts" that "Windows handle it better" ;)

So I made simple "dd" test, with RAW read and write without any file system.

 # sh -c "sync && echo 3 > /proc/sys/vm/drop_caches"  
 # dd of=/dev/sdc if=/dev/zero bs=64M oflag=dsync,direct iflag=fullblock & echo $!; while sudo kill -SIGUSR1 $!; do sleep 1; done  
 [1] 22786  
 22786  
 0+0 records in  
 0+0 records out  
 0 bytes copied, 0,00882205 s, 0,0 kB/s  
 1+0 records in  
 1+0 records out  
 67108864 bytes (67 MB, 64 MiB) copied, 1,82117 s, 36,8 MB/s  
 2+0 records in  
 2+0 records out  
 134217728 bytes (134 MB, 128 MiB) copied, 3,39514 s, 39,5 MB/s  
 3+0 records in  
 3+0 records out  
 201326592 bytes (201 MB, 192 MiB) copied, 5,00308 s, 40,2 MB/s  
 4+0 records in  
 4+0 records out  
 268435456 bytes (268 MB, 256 MiB) copied, 6,58802 s, 40,7 MB/s  
 5+0 records in  
 5+0 records out  
 335544320 bytes (336 MB, 320 MiB) copied, 8,37488 s, 40,1 MB/s  
 # killall dd  

 # sh -c "sync && echo 3 > /proc/sys/vm/drop_caches"  
 # dd if=/dev/sdc of=/dev/null bs=64M iflag=fullblock,dsync,direct & echo $!; while sudo kill -  
 [2] 31336  
 31336  
 1+0 records in  
 0+0 records out  
 0 bytes copied, 0,57976 s, 0,0 kB/s  
 2+0 records in  
 1+0 records out  
 67108864 bytes (67 MB, 64 MiB) copied, 1,13625 s, 59,1 MB/s  
 4+0 records in  
 3+0 records out  
 201326592 bytes (201 MB, 192 MiB) copied, 2,25124 s, 89,4 MB/s  
 6+0 records in  
 5+0 records out  
 335544320 bytes (336 MB, 320 MiB) copied, 3,36255 s, 99,8 MB/s  
 8+0 records in  
 7+0 records out  
 469762048 bytes (470 MB, 448 MiB) copied, 4,47168 s, 105 MB/s  
 10+0 records in  
 9+0 records out  
 603979776 bytes (604 MB, 576 MiB) copied, 5,58709 s, 108 MB/s  
 11+0 records in  
 10+0 records out  
 671088640 bytes (671 MB, 640 MiB) copied, 6,14454 s, 109 MB/s  
 13+0 records in  
 12+0 records out  
 805306368 bytes (805 MB, 768 MiB) copied, 7,25453 s, 111 MB/s  
 15+0 records in  
 14+0 records out  
 939524096 bytes (940 MB, 896 MiB) copied, 8,3586 s, 112 MB/s  
 17+0 records in  
 16+0 records out  
 1073741824 bytes (1,1 GB, 1,0 GiB) copied, 9,46021 s, 114 MB/s  

Looks like ~40MB/s for write and ~100MB/s for read. So .... yes, this is as advertised by SanDisk. You just need to look at those little stars placed near the numbers. You will find their meaning at the other side of the box. They claim 100MB/s for reading and up to 10x times of standard drive speed (4MB/s) for writing.

But I wonder how well my system behave when I would use this drive together with some file system. The most obvious choice is VFAT.
mkfs is your best friend. Let's use default parameters then.

 # mkfs.vfat /dev/sdc1  
 mkfs.fat 3.0.28 (2015-05-16)  
 # fsck.vfat /dev/sdc1 -v  
 fsck.fat 3.0.28 (2015-05-16)  
 Checking we can access the last sector of the filesystem  
 Boot sector contents:  
 System ID "mkfs.fat"  
 Media byte 0xf8 (hard disk)  
 512 bytes per logical sector  
 32768 bytes per cluster  
 32 reserved sectors  
 First FAT starts at byte 16384 (sector 32)  
 2 FATs, 32 bit entries  
 7580160 bytes per FAT (= 14805 sectors)  
 Root directory start at cluster 2 (arbitrary size)  
 Data area starts at byte 15176704 (sector 29642)  
 1894928 data clusters (62093000704 bytes)  
 32 sectors/track, 64 heads  
 2048 hidden sectors  
 121305055 sectors total  
 Checking for unused clusters.  
 Checking free cluster summary.  
 /dev/sdc1: 0 files, 1/1894928 clusters   
 # cd /media/rad/837C-93FB/  
 # dd of=./test1.txt if=/dev/zero bs=32M oflag=dsync mount=1000 & echo $!; while sudo kill -SIGUSR1 $!; do sleep 1; done  
 [1] 9897  
 9897  
 0+0 records in  
 0+0 records out  
 0 bytes copied, 0,0124351 s, 0,0 kB/s  
 0+1 records in  
 0+1 records out  
 18821120 bytes (19 MB, 18 MiB) copied, 1,0552 s, 17,8 MB/s  
 1+1 records in  
 1+1 records out  
 52375552 bytes (52 MB, 50 MiB) copied, 2,47131 s, 21,2 MB/s  
 2+1 records in  
 2+1 records out  
 85929984 bytes (86 MB, 82 MiB) copied, 4,14265 s, 20,7 MB/s  
 3+1 records in  
 3+1 records out  
 119484416 bytes (119 MB, 114 MiB) copied, 5,55161 s, 21,5 MB/s  
 4+1 records in  
 4+1 records out  
 153038848 bytes (153 MB, 146 MiB) copied, 7,0338 s, 21,8 MB/s  
 5+1 records in  
 5+1 records out  
 186593280 bytes (187 MB, 178 MiB) copied, 8,69431 s, 21,5 MB/s  
 6+1 records in  
 6+1 records out  
 220147712 bytes (220 MB, 210 MiB) copied, 10,1471 s, 21,7 MB/s  

What the f... ? ~21MB/s !!!!
So after a while I realized that this is probably due default sector size. Lets try to reformat for 4k sector size:

 # mkfs.vfat /dev/sdc1 -S 4096 -s 1  
 mkfs.fat 3.0.28 (2015-05-16)  
 # fsck.vfat /dev/sdc1 -v  
 fsck.fat 3.0.28 (2015-05-16)  
 Checking we can access the last sector of the filesystem  
 Boot sector contents:  
 System ID "mkfs.fat"  
 Media byte 0xf8 (hard disk)  
 4096 bytes per logical sector  
 4096 bytes per cluster  
 32 reserved sectors  
 First FAT starts at byte 131072 (sector 32)  
 2 FATs, 32 bit entries  
 60534784 bytes per FAT (= 14779 sectors)  
 Root directory start at cluster 2 (arbitrary size)  
 Data area starts at byte 121200640 (sector 29590)  
 15133546 data clusters (61987004416 bytes)  
 32 sectors/track, 64 heads  
 2048 hidden sectors  
 15163136 sectors total  
 Checking for unused clusters.  
 Checking free cluster summary.  
 /dev/sdc1: 0 files, 1/15133546 clusters  
 root@rad-desktop:/media/rad# cd  
 0AC2-20C2/ dysk/  
 root@rad-desktop:/media/rad# cd 0AC2-20C2/  
 root@rad-desktop:/media/rad/0AC2-20C2# dd of=./test.txt if=/dev/zero bs=64M oflag=dsync,direct iflag=fullblock & echo $!; while sudo kill -SIGUSR1 $!; do sleep 1; done  
 [1] 21689  
 21689  
 0+0 records in  
 0+0 records out  
 0 bytes copied, 0,0138234 s, 0,0 kB/s  
 1+0 records in  
 1+0 records out  
 67108864 bytes (67 MB, 64 MiB) copied, 2,02883 s, 33,1 MB/s  
 1+0 records in  
 1+0 records out  
 67108864 bytes (67 MB, 64 MiB) copied, 2,03412 s, 33,0 MB/s  
 2+0 records in  
 2+0 records out  
 134217728 bytes (134 MB, 128 MiB) copied, 3,77909 s, 35,5 MB/s  
 3+0 records in  
 3+0 records out  
 201326592 bytes (201 MB, 192 MiB) copied, 5,97208 s, 33,7 MB/s  
 4+0 records in  
 4+0 records out  
 268435456 bytes (268 MB, 256 MiB) copied, 7,93711 s, 33,8 MB/s  
 5+0 records in  
 5+0 records out  
 335544320 bytes (336 MB, 320 MiB) copied, 9,74674 s, 34,4 MB/s  
 6+0 records in  
 6+0 records out  
 402653184 bytes (403 MB, 384 MiB) copied, 11,8841 s, 33,9 MB/s  
 7+0 records in  
 7+0 records out  
 469762048 bytes (470 MB, 448 MiB) copied, 13,8475 s, 33,9 MB/s  
 ^C  

35MB/s ... hmmm. So the reason for this is native sector size of the flash drive. It seems that the device advertise 512B as native sector size but it internally use 4k. Because of this each write operation need to be combined in the firmware of the drive which degrades the performance.
I was to lazy to check, what the mkfs.vfat tools would do if drive would report 4k sectors. I suppose that it should use bigger FAT sectors but that's only a wild guess (or rather a wish).

Now, you probably wonder... How many of wrongly formatted flash drives are in your possession? ;)

Mapping device memory with mmap and MAP_PRIVATE on /dev/mem

From time to time you hit into something, which proves that even the old dog can learn new things.
Did you ever need to mmap() the device memory?

I recently work on one of the implementations of ODP (Dataplane framework similar to DPDK) and naturally I need to mmap of device or physical address space into userspace process.

Mapping of PCI BAR0, simple stuff right?

addr = mmap (NULL, BAR0_SIZE, PROT_READ, MAP_PRIVATE, mem_fd, BAR0_OFFSET);

so why I use MAP_PRIVATE and what is wrong with it?

Usually when you map piece of memory and you do not want to share the memory page with other processes, you use MAP_PRIVATE. It's fully legitimate with typical memory as when in conjunction with PROT_READ, kernel use copy-on-write and allocate the page when it is first time referenced.

Wait... what I just said? It is copied ...
So remember, never use MAP_PRIVATE when mapping PCI BARS, because it will COPY the BAR memory. In other words references to this mapped memory will not reach the PCI. You will not operate on device address space anymore, instead you will operate on copy of PCI address space inside of RAM.

Why WaitForMultipleObjects() is bad and why useful things seems to get lost in dark corners of internet much easier than pictures of funny cats

Some time ago, I made a small research of embedded RTOS. I also published my personal opinion about them in my second blog.

Recently I'm preparing a newspaper article and just wanted to recall some interesting point of view why Microsoft Windows WaitForMultipleObjects() functions family is a bad idea. And guess what ... I spend almost hour to find this post in the internet ... ughh

So since things like to get lost in dark corners o internet ... I insolently decided to copy-paste this post :D ... the original post can be (as time of writing this note) found here.


Author of following post is  by Kaz Kylhe Mon, 19 Apr 1999 04:00:00
Original thread name: "pthreads and waiting for MULTIPLE conditions"
Site: http://www.verycomputer.com/5_c6341d98f68da45e_1.htm


Andy Levin was asking for:

I am porting some NT work I wrote a while back to the UNIX world and have the need to use pthreads.
Is there any functionality in the pthread libraries (or elsewhere) that allows you to wait for MULTIPLE conditions to occur.

Kaz Kylhe replays:

This is not needed. Think about it. You have one thread and you want to
put it to sleep. In a typical OS kernel, all processes or threads sleep
in one ``place'' at a time, such as a supend queue and whatnot.

What you are asking for is like wanting to sleep in two or more beds
at the same time. This, of course, is only possible if you are the president of
the United States.

> I am looking for functionality similar to the Win32 function WaitForMultipleObjects()

That Win32 function is seriously misconceived and should be avoided even in
Win32 programming. It has problems. For example, in the case that any *one* of
the objects can wake up the thread, there is potential starvation by one object
that is constantly going off, since the function only reports one object during
one call. (Contrast this to the much cleverer POSIX select() function that
gives you a bitmap representing objects that are ready, thus avoiding
starvation.) Another problem is that this function does not scale beyond 64
objects. If you need to wait for more events or what have you, you have to
launch additional threads. For example, if you wanted to wait for 256 objects
in Win32, you would launch four threads. Each thread would wait for 64
objects. And your ``parent'' thread would do a WaitForMultipleObjects on the
four thread handles. Ugly, ugly, ugly.

Programs that use WaitForMultipleObjects often break encapsulation; you often
see a technique whereby software modules export events, in effect saying ``here
is my event, I will set it when something interesting happens''. Needless
to say, this is hard to port.

A much better approach is to keep events buried in the implementation of
something; notification can be provided much more efficiently by direct message
passing. One object should not know about the internal events and threads of
another object.

For example, suppose that you have a thread that needs to wait until some other
object completes some task, or a timer goes off. One way to do this under Win32
might be to have two events. Your thread waits on both of them using
WaitForMultipleObjects. The events are exported to other objects (such as the
timer or the worker) which do a SetEvent. In other words, the use of Win32
events is made explicit in the interfaces between objects, a clearly
portability mistake.

A better way to do this might be to have registered callbacks. When the timer
goes off, a callback is invoked back to the waiting object. When the worker
object is done doing something, a different callback is invoked. Both
callbacks can signal the same Win32 event, so your thread just needs to wait on
that single one. This model is readily supported by POSIX condition variables,
and is easier to port among different threading platforms, because it is based
on a message passing abstraction that is directly supported by C and C++,
namely function calls.


>allows you to specify an array of synchronization objects and conditions wait for all, wait for any, etc) to wait for.

There is no etc: you can wait for all, or you can wait for any (with
potential starvation, requiring you to constantly permute the array
passed to WaitForMultipleObjects).