2024-06-23.log

sevanfor some reason dkms process ends up building the zfs module twice00:24
sevanfirst as part of the initial process to generate the new module then again on the installation attempt.00:25
sevanhaven't investigated but perhaps the package maintainer has set it so it will always be built from source for whatever operation? (never make use of anything prepared)00:26
sevananyway, new kernel installed, rebooting.00:27
joschfingers crossed!00:27
joschsevan: there was a reason but i forgot -- i think it was the same reason as to why the initramfs is built twice00:27
sevanjosch: if the system has booted the display hasn't initialised00:30
sevanjosch: re initramfs, ah ok, I see that too00:30
joschsevan: could you connect with a uart adapter to see what is printed via serial?00:30
sevanye00:31
sevanp00:31
sevanneed to find my adapters00:32
+ chomwitt (~chomwitt@ppp-2-84-146-212.home.otenet.gr)00:45
joschsevan: do you have an estimate? i wonder whether i should go to bed or wait a bit longer :)00:59
sevanjosch: sorry01:00
sevanjosch: starting kernel & it just hangs there01:01
sevanjosch: http://paste.debian.net/1321205/01:01
joschokay, that's the same failure mode as on the pocket01:02
sevanok01:02
joschin some way this relieves me a bit because it would've been weird if only the pocket was affected01:02
sevanack01:03
sevanso keep eyes peeled here for a new kernel announcement?01:03
joschnot really01:04
joschwant to try out 6.9?01:04
sevan:D01:04
joschthe failure happened on the *same* kernel version01:04
sevansure01:04
joschthe only difference was, when it was built01:04
joschso it must be something in the build environment that makes the difference01:04
joschi tried comparing the vmlinuz that worked with the one that did not01:05
joschmaybe notably, the working vmlinuz is 31M large and the failing one only 26M01:05
joschso something in the build environment in the last 3 weeks broke boot and affected the kernel size01:05
sevanerm, that's a big difference01:06
joschyup01:06
joschbut it could also be a red herring01:06
joschand maybe the size difference is weird but unrelated01:06
sevanmaybe we can try tomorrow, there's going to be a 40+ minute turnaround because of the dkms thingy.01:06
sevanwith the 6.9 kernel. ^01:07
joschsevan: sure, you find the 6.9 kernel as a CI artifact01:07
sevanok01:07
sevanbtw I see the size difference01:08
sevan-rw-r--r--  1 root root  30M Mar  9 23:22 vmlinuz-6.7.9-reform2-arm6401:08
sevan-rw-r--r--  1 root root  26M Jun  5 11:42 vmlinuz-6.8.12-mnt-reform-arm6401:08
joschthose are different versions but yes01:08
sevanwe have a single kernel for reform2 & pocket?01:08
joschwe have a single kernel for everything, yes01:08
sevanok, must be the magic of dedupe :)01:09
jfredThis sort of thing is why I'm happy that Guix tightly controls the build environment for packages so much... I still need to try out that Guix install on my Reform01:16
joschjfred: oh i can also look up which packages in the build environment different between the successful and failing build, thanks to reproducible builds01:17
joschusing the buildinfo file, the debian linux kernel is bit-by-bit reproducible01:18
joschso i have no problem reproducing the failing and the successful kernel01:18
joschbut that still does not tell me what is going on01:18
chbinutils-arm-linux-gnueabihf 2.42-4 -> 2.42.50.20240618-1 ?01:18
joschmaybe that01:19
joschor maybe debhelper01:19
joschi have to try out some variations tomorrow01:19
chhttps://tracker.debian.org/news/1537738/accepted-debhelper-1316-source-into-unstable/ seems unlikely01:22
joschindeed01:22
joschon the CI i triggered a run using testing as the base which has the same binutils but different debhelper -- trying out that gives us a few more data points and package versions to compare01:23
joschi'll do a local build over night with binutils-arm-linux-gnueabihf 2.42-401:24
joschi guess you are right that this is the most likely culprit01:24
joschespecially given the file size difference as well...01:25
chsizes in debians 6.8.12-1 vs 6.9.2-1~exp1 also look suspicious01:25
joschch: in what way do you mean?01:25
ch6.9.2-1~exp1 vmlinuz is 4M smaller01:25
joschright, but if that would be unbootable there would definitely be a bug open about that already01:26
chits in experimental01:26
joschch: i followed the development of 6.9 in salsa including successful test reports01:28
joschunless of course they built this on bookworm which also is unlikely :)01:28
joschanyways, must test more :)01:28
chbut did you follow the binutils development?01:28
ch6.9.2-1~exp1 looks pretty much broken to me01:28
joschoh?01:29
joschdoes somebody know someone with an x86 box who would be willing to test 6.9.2-1~exp1? :)01:30
chwill do in a sec01:30
joschuh nice01:30
joschsleep delayed :D01:30
jfredjosch: Ahh right! Makes sense re: reproducible builds01:30
charm64 was easier01:31
joschch: what arm64 system do you have that uses the vanilla kernel?01:31
- chomwitt (QUIT: Ping timeout: 256 seconds) (~chomwitt@ppp-2-84-146-212.home.otenet.gr)01:31
chutm on mac01:31
chcheating, i know01:32
joschheh :)01:32
joschemulating x86 boot on my a311d reform is so slow that things time out left right and center during boot XD01:32
josch(using qemu)01:32
chyeah dealing with the physical x86 hardware is also slower and annoying :)01:33
chamd64 boots01:35
chwow, the broken 6.9 managed to crash qemu/utm01:36
joschoh?01:36
joschi'm now trying this locally for arm64 using debvm as well01:37
joschi didn't even consider that it could be broken for everybody and not just for us...01:38
joschch: same results for arm64 under qemu over here01:40
+ chomwitt (~chomwitt@2a02:587:7a1d:6300:1ac0:4dff:fedb:a3f1)01:44
chjosch: are you going to file a bug against binutils + linux-image-6.9-arm64?01:45
chnot sure i can add any useful detail01:45
joschch: we do not know yet whether it is binutils though01:46
joschi already wrote quite a bit of text in #debian-kernel over on OFTC01:46
joschif i do not get any responses there, i'll file a bug, yes01:46
chhm, new binutils is already in testing01:47
chno urgency then :D01:47
joschexactly01:49
joschch: thanks a lot for your input -- without your insistence i would not've evaluated whether vanilla debian kernel exhibits the same issue or not01:49
joschbut now i'm happy and can go to sleep with a good feeling :)01:49
chsleep well!01:49
joschthank you! :)01:50
+ hjcfd (~hjcs@syn-098-015-248-249.res.spectrum.com)01:53
joschch: why must i only learn now that you are zeha!!! XD01:53
chhaha01:53
josch(yes, the nick name is funny)01:53
chtoo obvious!01:53
- ZylonMaster (QUIT: Ping timeout: 268 seconds) (~hjcs@syn-098-015-248-249.res.spectrum.com)01:56
- hjcfd (QUIT: Quit: Leaving) (~hjcs@syn-098-015-248-249.res.spectrum.com)02:03
- mtm (QUIT: Ping timeout: 264 seconds) (~mtm@c-71-228-84-213.hsd1.fl.comcast.net)02:03
- mjw (QUIT: Ping timeout: 252 seconds) (~mjw@gnu.wildebeest.org)02:06
- chomwitt (QUIT: Ping timeout: 268 seconds) (~chomwitt@2a02:587:7a1d:6300:1ac0:4dff:fedb:a3f1)03:18
- nsc (QUIT: Ping timeout: 264 seconds) (~nicolas@i5C74DF86.versanet.de)03:23
+ nsc (~nicolas@i5C74DC29.versanet.de)03:24
- eschaton (QUIT: Remote host closed the connection) (eschaton@2600:3c01::f03c:91ff:fefd:5d92)03:51
+ eschaton (eschaton@2600:3c01::f03c:91ff:fefd:5d92)03:51
+ mtm (~mtm@c-71-228-84-213.hsd1.fl.comcast.net)03:59
- mtm (QUIT: Ping timeout: 255 seconds) (~mtm@c-71-228-84-213.hsd1.fl.comcast.net)05:56
- cobra (QUIT: Ping timeout: 256 seconds) (~cobra@user/Cobra)06:19
+ cobra (~cobra@user/Cobra)06:28
- Gooberpatrol66 (QUIT: Quit: Konversation terminated!) (~Gooberpat@user/gooberpatrol66)07:04
+ Gooberpatrol66 (~Gooberpat@user/gooberpatrol66)07:20
- cobra (QUIT: Ping timeout: 268 seconds) (~cobra@user/Cobra)07:21
+ chomwitt (~chomwitt@2a02:587:7a1d:6300:1ac0:4dff:fedb:a3f1)08:02
+ mtm (~mtm@c-71-228-84-213.hsd1.fl.comcast.net)08:26
- chomwitt (QUIT: Ping timeout: 260 seconds) (~chomwitt@2a02:587:7a1d:6300:1ac0:4dff:fedb:a3f1)08:35
- mtm (QUIT: Ping timeout: 252 seconds) (~mtm@c-71-228-84-213.hsd1.fl.comcast.net)08:53
joschch: i found your theory that it could be binutils very convincing and prepared a MR which chooses old-enough binutils (and gcc-13 as new gcc does not work with old binutils) from snapshot.d.o: https://source.mnt.re/reform/reform-debian-packages/-/merge_requests/4309:18
joschif somebody who wants to help out and for whom their reform is not their main machine wants to test this: feel free to try and install the linux-image that falls out of this pipeline09:19
- megasquid (QUIT: Ping timeout: 268 seconds) (~megasquid@user/megasquid)09:27
chjosch: i'm guessing the reform kernel should also boot in qemu? if so then this could also be a good first test?10:19
joschyes, it's on my todo list but it's family time today :)10:33
joschch: and yes, we have a test script using tcl-exect which tests the kernel in qemu :)10:34
- nybble (QUIT: Ping timeout: 246 seconds) (~sprang@user/csprng)10:36
ch:)10:37
joschuuuuh would you look at that!10:38
joschwith old binutils, the kernel size is 31 MB10:39
joschstep aside daughter, daddy needs some time with qemu10:39
chnot gonna tell you what to do, but i know time with children is precious10:40
joschi blame Lukas for having made this laptop ;)10:41
joschyes! it boots in qemu!11:03
joschnow, just to be sure, let me create a system image with the non-working kernel packages -- just to verify my test setup11:04
joschconfirmed!!11:43
joschch: thank you!!11:43
chnice11:43
+ mjw (~mjw@gnu.wildebeest.org)11:45
joschit's #1074111 now11:53
joschminute: the boot problem has a workaround now in this merge request: https://source.mnt.re/reform/reform-debian-packages/-/merge_requests/4311:55
joschthis makes it work in qemu and the kernel size is good again, so i'm quite confident that this also fixes it on the real hardware11:55
joschstill, it would be nice if somebody could try this out for real using this package: https://source.mnt.re/reform/reform-debian-packages/-/jobs/4686/artifacts/file/repo/pool/main/l/linux/linux-image-6.8.12-mnt-reform-arm64_6.8.12-1+reform20240623T071545Z_arm64.deb11:56
joschthanks in advance!11:56
joschnow finally out and afk for the rest of the day11:56
josch_o/11:56
+ cobra (~cobra@user/Cobra)12:18
+ reform25278 (~user@business-90-187-186-49.pool2.vodafone-ip.de)13:18
- sevan (QUIT: Ping timeout: 272 seconds) (~sevan@user/venture37)13:18
reform25278https://pastebin.com/wGwiDCpn13:18
reform25278rebooting now13:18
- reform25278 (QUIT: Remote host closed the connection) (~user@business-90-187-186-49.pool2.vodafone-ip.de)13:18
pandoraWorked13:20
pandoraBuuut13:36
pandoraI don’t have network anymore13:36
chlike, no eth interface or it doesn't work? which nic?13:39
chi wonder what 'Starting background process to update reform-qcacld2' actually does13:40
pandoraNo eth13:41
pandoraOnly the loopback and end1 . Guess the latter is the SIM card13:41
chand your eth should be the qcacld2 thing?13:41
pandoraIdk tbh13:42
pandoraI got the reform13:42
pandoraA few days back13:42
pandoraI will upload a dmesg in a minute13:44
choO( my sane resistance to buy a pocket reform is waning every minute )13:49
pandoracan't find a usb-c stick rn :D13:50
chmaybe other q: your eth is supposed to be wired or wifi?13:51
ch(qcacld2 is the wifi driver)13:51
+ mtm (~mtm@c-71-228-84-213.hsd1.fl.comcast.net)13:51
pandorawifi13:53
pandorabut can't see either of the interfaces13:53
pandorahere is the dmesg log13:55
pandorahttps://pastebin.com/tXe75R0F13:55
chminute: is there a way of getting a reform purple + sleeve with an rk3588 cpumodule without getting the imx8 module and then swapping it?14:01
- mtm (QUIT: Ping timeout: 240 seconds) (~mtm@c-71-228-84-213.hsd1.fl.comcast.net)14:03
+ akira (~akira@2a01:599:a0e:73eb:1c49:80f0:acf9:2709)14:27
Zabaend1 is what eth0 gets renamed to15:11
- akira (QUIT: Read error: Connection reset by peer) (~akira@2a01:599:a0e:73eb:1c49:80f0:acf9:2709)15:21
+ akira (~akira@ipb218fb58.dynamic.kabel-deutschland.de)15:22
- akira (QUIT: Read error: Connection reset by peer) (~akira@ipb218fb58.dynamic.kabel-deutschland.de)15:24
+ akira (~akira@ipb218fb58.dynamic.kabel-deutschland.de)15:24
- akira (QUIT: Ping timeout: 256 seconds) (~akira@ipb218fb58.dynamic.kabel-deutschland.de)15:31
pandorayeah u r right ... but i can't test that because i don't have an industrial ethernet cable / adapter 15:31
+ akira (~akira@2a01:599:a0e:73eb:1c49:80f0:acf9:2709)15:32
+ sevan (~sevan@2001:470:1f1d:1d6:5a55:caff:fe24:ed4)15:39
- sevan (QUIT: Changing host) (~sevan@2001:470:1f1d:1d6:5a55:caff:fe24:ed4)15:39
+ sevan (~sevan@user/venture37)15:39
- akira (QUIT: Read error: Connection reset by peer) (~akira@2a01:599:a0e:73eb:1c49:80f0:acf9:2709)15:44
+ akira (~akira@178.24.251.58)15:45
- akira (QUIT: Ping timeout: 264 seconds) (~akira@178.24.251.58)15:56
+ akira (~akira@2a01:599:a0e:73eb:1c49:80f0:acf9:2709)16:00
+ mtm (~mtm@c-71-228-84-213.hsd1.fl.comcast.net)16:09
- akira (QUIT: Read error: Connection reset by peer) (~akira@2a01:599:a0e:73eb:1c49:80f0:acf9:2709)16:15
+ akira (~akira@2a01:599:a0e:73eb:1c49:80f0:acf9:2709)16:16
- akira (QUIT: Read error: Connection reset by peer) (~akira@2a01:599:a0e:73eb:1c49:80f0:acf9:2709)16:22
+ akira (~akira@37.4.230.225)16:22
+ chomwitt (~chomwitt@2a02:587:7a27:9000:1ac0:4dff:fedb:a3f1)16:31
minutepandora: end1 is ethernet17:20
minuteqcacld2 is the wifi driver17:20
noam...I don't think I've ever used the ... key to the right of ctrl on the reform, lol17:36
noami think i semiregularly realize that :P17:36
truckI remapped it to ctrl, because it was in the wrong place for my fingers, and there was no other use for it17:41
truckI think it's there for folks who are used to shift lock being there (I've seen .de layouts with both caps and shift lock)17:42
truckand, of course, for keeping a nice orthargonal layout (:17:42
pandora@minute: interesting. I don’t a see any mentions of qcacld2 in dmesg17:59
noamliterally no idea what it's for lol18:03
+ nybble (~sprang@user/csprng)18:57
- cobra (QUIT: Ping timeout: 256 seconds) (~cobra@user/Cobra)19:20
jfredSomething odd I've noticed... when I plug my ebook reader (Kobo, forget which model) into my Reform while it's charging, the Kobo's touchscreen goes unresponsive until I unplug the Reform from power. Weird grounding thing maybe?19:30
pandoraIs it a 2 or 3 prong connector in the power plug?19:53
pandoraI think there is a long forum post on this19:53
pandoraOther laptops with 2 prong power supply behave similar. On some u can get a tingling feeling if the laptop is fully metal19:54
pandorahttps://community.mnt.re/t/psu-not-suitable-for-use-with-metal-housing-resolved/1713/319:56
+ cobra (~cobra@user/Cobra)20:07
noamum20:26
noamshould i be concerned if the keyboard light shut off out of nowhere? >_<20:27
noameverything else is totally fine lol20:27
- akira (QUIT: Ping timeout: 252 seconds) (~akira@37.4.230.225)20:50
+ akira (~akira@2a01:599:a2b:1eff:c85d:fd50:5b36:418)20:52
- akira (QUIT: Read error: Connection reset by peer) (~akira@2a01:599:a2b:1eff:c85d:fd50:5b36:418)20:59
+ akira (~akira@81.92.17.144)20:59
- akira (QUIT: Ping timeout: 268 seconds) (~akira@81.92.17.144)21:03
+ akira (~akira@2a01:599:a2b:1eff:c85d:fd50:5b36:418)21:05
minutepandora: take a look at /sbin/reform-hw-setup and search for qcacld2, then you'll see how that is loaded and probably not found in your case21:27
pandorait says insmod -f "/opt/reform/qcacld2/qcacld2-${KERNVER}.ko"21:37
pandoraif i execute that manually the wifi starts to work21:39
pandorafor some reason the "automatic" part broke after I installed 21:39
pandorahttps://source.mnt.re/reform/reform-debian-packages/-/jobs/4686/artifacts/file/repo/pool/main/l/linux/linux-image-6.8.12-mnt-reform-arm64_6.8.12-1+reform20240623T071545Z_arm64.deb21:39
pandorai thik the issue is ${KERNVER}21:43
pandorait is different than what i find in /opt/reform/qcacld2/21:44
+ reform12450 (~user@business-90-187-186-49.pool2.vodafone-ip.de)21:45
reform12450user@mnt:~$ dpkg-query --show --showformat '${Version}' "linux-image-$(uname -r)"21:45
reform124506.8.12-1+reform20240623T071545Zuser@mnt:~$ ls /opt/reform-qcacld2/21:45
reform12450qcacld2-6.8.12-1+reform20240605T104210Z.ko21:45
- akira (QUIT: Read error: Connection reset by peer) (~akira@2a01:599:a2b:1eff:c85d:fd50:5b36:418)22:14
joschpandora: /sbin/reform-hw-setup does not use set -e, so if parts of it fail, the script will not fail22:15
joschpandora: you might be interested in helping with improving https://source.mnt.re/reform/reform-tools/-/merge_requests/7022:16
joschin your case it sounds like you are missing an upgrade to the latest qcacld2 package22:17
joschpandora: can you also show me the output of this: apt-cache policy reform-qcacld2-6.8.12-mnt-reform-arm6422:21
- reform12450 (QUIT: Remote host closed the connection) (~user@business-90-187-186-49.pool2.vodafone-ip.de)22:22
+ reform12631 (~user@90.187.186.49)22:29
reform12631user@mnt:~$ apt-cache policy reform-qcacld2-6.8.12-mnt-reform-arm64 22:29
reform12631reform-qcacld2-6.8.12-mnt-reform-arm64:22:29
reform12631  Installed: 20240605T114456Z22:29
reform12631  Candidate: 20240605T114456Z22:29
reform12631  Version table:22:29
reform12631 *** 20240605T114456Z 10022:29
joschoh right, because your manually installed kernel does not come from any repo22:30
pandorayeah probably 22:31
joschreform12631: the situation regarding reform-qcacld2 is quite unfortunate. I'd call the current situation a hack but i do not know how to improve on it22:31
joschthe fundamental problem is that qcacld2 exists in the first place22:31
joschideally, the driver would be in-tree and then we'd not be having this problem22:31
pandorawouldn't that be something that should be upstreamed in the kernel?22:36
joschyes22:36
joschif you want to work on that please do!22:36
pandorauntil then wouldn't it be better to just load the latest available version of qcacld2?22:37
joschpandora: you mean as a dkms module?22:37
pandoranot but use something like this:22:39
pandoralatest_driver=$(ls /opt/reform-qcacld2/qcacld2-*.ko | sort -V | tail -n 1)22:39
pandorainsmod -f "$latest_driver"22:39
pandorato make sure the highest version is loaded22:40
joschpandora: and what do you do if the currently running kernel is not equal to the latest version you find in /opt/reform-qcacld2/?22:40
pandorau could try to load the exact match if there is any otherwise load the highest availale version22:40
joschyes, a fallback mechanism if there was no match is something we could do22:40
joschpandora: would you like to try writing a patch that does that?22:41
joschpandora: alternatively, would you like to receive and test a diff i send you?22:41
joschpandora: you could even make it such that it iterates through all /opt/reform-qcacld2/qcacld2-*.ko files in decending order until it finds one that succeeds to load, no?22:42
pandorayes i could do that. not sure about a "patch". normally at work i just create a branch. fix something and create a mr22:43
pandorabut i do machine learning normally :D22:43
joschpandora: you can also submit a MR -- do you have an account at source.mnt.re?22:44
pandoralmc22:45
pandoraits waiting for approval22:46
joschLukas is currently on holidays, so we may need to go the patch route :)22:46
joschjust run "git format-patch -1" once your commit is ready and put the resulting file into a pastebin or send it by mail22:46
pandoralet me clone the repo and try :D22:48
joschpandora: i would be very happy (but this is not a requirement) if you also submit a fix for the pocket reform part of https://source.mnt.re/reform/reform-tools/-/merge_requests/70 as part of your patch22:49
joschby having your patch do "set -e" at the top or in the pocket branches22:49
+ akira (~akira@2a01:599:a2b:1eff:c85d:fd50:5b36:418)22:50
- akira (QUIT: Read error: Connection reset by peer) (~akira@2a01:599:a2b:1eff:c85d:fd50:5b36:418)22:51
pandoralet me try to write a patch for it first :D22:51
+ akira (~akira@2a01:599:a2b:1eff:c85d:fd50:5b36:418)22:54
- akira (QUIT: Read error: Connection reset by peer) (~akira@2a01:599:a2b:1eff:c85d:fd50:5b36:418)23:00
+ akira (~akira@2a01:599:a2b:1eff:c85d:fd50:5b36:418)23:07
- akira (QUIT: Read error: Connection reset by peer) (~akira@2a01:599:a2b:1eff:c85d:fd50:5b36:418)23:08
+ akira (~akira@2a01:599:a2b:1eff:c85d:fd50:5b36:418)23:10
- akira (QUIT: Read error: Connection reset by peer) (~akira@2a01:599:a2b:1eff:c85d:fd50:5b36:418)23:10
- reform12631 (QUIT: Ping timeout: 246 seconds) (~user@90.187.186.49)23:11
- mjw (QUIT: Ping timeout: 264 seconds) (~mjw@gnu.wildebeest.org)23:18
+ akira (~akira@2a01:599:a2b:1eff:c85d:fd50:5b36:418)23:24
+ jacobk (~quassel@2600:1700:9e1e:7800:2c94:5a14:bbc3:b280)23:31
pandoraok wrote a little patch and it seems to work on my reform23:38
minutepandora: what's your username on source? sorry for delays, but we only let in people after they ask here or via support email23:53
minute(to not have spam repos etc)23:53
minute(yes i'm on vacation but can click that button later)23:54
pandorapandora23:55
pandora@josch: send you a patch via mai23:55
pandoramail23:55
pandora"It works on my machine"23:56
pandorait contains the set -e too23:56
pandoratho i am not sure if that is really a good idea. if something fails it will just exit i guess without trying to do all the other stuff in the script23:58
joschpandora: with set -e you have to do correct error handling23:58
minutepandora: acc approved23:58
pandoramerci23:58

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!