| vkoskiv | What I've been doing is just umounting /boot after boot and then yanking the card out | 00:02 |
|---|---|---|
| vkoskiv | Not sure if that's a correct thing to do | 00:02 |
| vkoskiv | I hit circle+x and now my keyboard is frozen | 00:07 |
| vkoskiv | (I admit, I was browsing the firmware code and wondered what that option does :D) | 00:07 |
| kfx | vkoskiv: my favorite Stupid Keyboard Trick is circle-0 on the usb standalone keyboard | 00:07 |
| vkoskiv | Not sure how to fix this | 00:08 |
| vkoskiv | I can't even turn the thing off | 00:08 |
| kfx | vkoskiv: ssh into it? | 00:08 |
| vkoskiv | Not enabled _D | 00:08 |
| vkoskiv | :D | 00:08 |
| vkoskiv | I'll plug a usb keeb in | 00:08 |
| vkoskiv | Rebooted. Yanked the battery cables to fix it | 00:11 |
| - Gooberpatrol66 (QUIT: Quit: Leaving) (~Gooberpat@user/gooberpatrol66) | 00:15 | |
| + vagrantc (~vagrant@2600:3c01:e000:21:7:77:0:20) | 00:48 | |
| minute | circle-x is enter USB flashing mode | 00:53 |
| minute | you can't leave that mode without resetting the keyboard or flashing a firmware yeah | 00:54 |
| - chomwitt (QUIT: Ping timeout: 248 seconds) (~chomwitt@2a02:587:dc1f:5600:518c:c264:c8fd:1f6b) | 01:00 | |
| Asmadeus | nice keyboard firmwares improvement btw, I've updated mine recently from last year's version (it's been a while!) and have been quite happy with it (layers and that flashing feature) | 01:01 |
| Asmadeus | didn't check with the reform version but for the standalone one some media keys aren't getting through though, need to check why/what's going wrong | 01:01 |
| - mjw (QUIT: Quit: Leaving) (~mark@gnu.wildebeest.org) | 01:19 | |
| - Christoph_ (QUIT: Remote host closed the connection) (~Christoph@p54bf61d7.dip0.t-ipconnect.de) | 01:20 | |
| + klardotsh (~klardotsh@172.56.104.60) | 01:53 | |
| klardotsh | howdy folks. anyone manage to evidently brick their displays on day two of owning their Reforms? because that's evidently where I'm at. popped the back off to install a wifi antenna, reassembled, and now I get screen backlight but no actual content on screen, on any OS (I've reflashed the SD a few times) | 01:54 |
| klardotsh | popped it back open, removed wifi antenna, no change | 01:54 |
| klardotsh | I suppose it's possible I shorted something, though I have no evidence to suggest that so far | 01:55 |
| - mtm (QUIT: Ping timeout: 240 seconds) (~mtm@c-73-27-62-116.hsd1.fl.comcast.net) | 02:03 | |
| - erle (QUIT: Ping timeout: 260 seconds) (~erle@ip5f5af7b3.dynamic.kabel-deutschland.de) | 02:04 | |
| + erle (~erle@ip5f5af7b3.dynamic.kabel-deutschland.de) | 02:09 | |
| kfx | klardotsh: do you have a serial cable? | 02:38 |
| kfx | e.g. https://www.adafruit.com/product/954 or similar | 02:38 |
| klardotsh | hmmmmmmm somewhere, probably. I'd have to go digging for it, and I'm not certain of what pins are on the device end, but probably. | 02:42 |
| + bkeys1 (~Thunderbi@static-198-54-135-69.cust.tzulo.com) | 02:44 | |
| kfx | klardotsh: that would at least tell you if the thing is starting at all | 02:45 |
| - bkeys (QUIT: Ping timeout: 248 seconds) (~Thunderbi@static-198-54-135-69.cust.tzulo.com) | 02:47 | |
| * bkeys1 -> bkeys | 02:47 | |
| klardotsh | I could have sworn I had at least two FDTI serial adapters in this house. what the hell box could they have ended up in.... | 04:00 |
| klardotsh | ACTION uproots entire house, realizing he could probably instead just bitbang this out on a RP2040 in half the time | 04:00 |
| + mtm (~mtm@c-73-27-62-116.hsd1.fl.comcast.net) | 04:09 | |
| - bkeys (QUIT: Remote host closed the connection) (~Thunderbi@static-198-54-135-69.cust.tzulo.com) | 04:13 | |
| - vagrantc (QUIT: Quit: leaving) (~vagrant@2600:3c01:e000:21:7:77:0:20) | 05:08 | |
| mtm | klardotsh: did you maybe jostle the display connector on the main board? | 05:14 |
| klardotsh | mtm: I've reseated it twice now since doing the antenna connect | 05:22 |
| klardotsh | unless I sheared a cable inside the hot glue, that cable should be fine | 05:22 |
| klardotsh | getting backlight especially makes me think it might be alright | 05:22 |
| klardotsh | (er, I guess that only means the backlight pins are okay....) | 05:23 |
| klardotsh | alright, I had exactly two FDTI serial adapters in the house. first one was DOA (never showed up in dmesg on the desktop), second one shows up. let's see what this bad boy says. | 05:42 |
| klardotsh | mtm: kfx: I'm at a Debian login prompt over UART (system image v3; though the "no display" issue was same on NixOS) | 05:44 |
| klardotsh | brightnessctl works too | 05:46 |
| klardotsh | [ 10.092027] ti_sn65dsi86 3-002c: [drm:ti_sn_bridge_enable [ti_sn65dsi86]] *ERROR* Link training failed, link is off (-5) | 05:55 |
| klardotsh | ah that seems bad | 05:55 |
| + Gooberpatrol66 (~Gooberpat@user/gooberpatrol66) | 05:56 | |
| kfx | I wonder if it's booting with a wrong device tree or something | 06:07 |
| klardotsh | The bigger question to me in that case would be "how?". Prior to the "sudo poweroff" to put the wifi antennae on I was rocking a NixOS SD card and just getting ready to set up the ZFS root on NVME at TTY. next boot, no display (despite same SD contents) | 06:11 |
| klardotsh | and now this failure is on the sys image v3 from gitlab | 06:11 |
| klardotsh | I wouldn't expect to see two OSes each with wrong DTBs, though nothing's impossible | 06:11 |
| kfx | have you ever moved the switch under the heatsink? | 06:32 |
| kfx | I basically immediately started playing games with which boot files the system sources from which storage medium, so I've painted myself into a corner before | 06:32 |
| klardotsh | I tried that this evening to try to boot the recovery OS at one point to see if it would do anything differently (it didn't, I put the switch back and reinstalled the heatsink) | 06:32 |
| kfx | ok | 06:32 |
| kfx | looks like that process is nondeterministic at best : https://lore.kernel.org/lkml/20201002135920.1.I2adbc90b2db127763e2444bd5a4e5bf30e1db8e5@changeid/T/ | 06:34 |
| kfx | have you tried disconnecting the batteries and power and bringing it up from a cold boot again? | 06:35 |
| klardotsh | hmmmm I've disconnected the batteries and power at various points, but I suppose I haven't done anything to explicitly drain the caps, if that makes any difference here | 06:36 |
| klardotsh | lemme give this thing the ol Circle-0 and disconnect everything for a min | 06:37 |
| klardotsh | one more thing to note, I just now tried a reboot with no wireless card plugged in. no difference, so it doesn't appear to be the case of RF interference described on the forums | 06:38 |
| kfx | when you reseated the video cable did you reseat both ends? | 06:39 |
| klardotsh | hmmmm no, I've never even seen the other end of the cable tbqh | 06:39 |
| klardotsh | should I go ahead and rip the lid enclosure apart for kicks? | 06:39 |
| kfx | I dunno, part of me thinks if it worked before it should still work and the other part of me disassembles the planet to double check | 06:41 |
| kfx | another choice is to crank up loglevel=7 in the kernel command line, and see if you get useful information near that link training failure | 06:42 |
| klardotsh | if I had time this weekend to just totally make a wreck of my house I'd find the mobo schematics in kicad and start toying around with a multimeter to make sure I didn't fry anything in the disp accel or gpu links somehow | 06:42 |
| klardotsh | there's a miniscule but possible chance I zapped something, IIRC (it's been hours so I don't fully remember) I may have left the batts connected when plugging in antennae because "well, it's antennas dude, I'm not doing NVME/PCIe work" | 06:43 |
| klardotsh | hm. is there a way to intercept uboot over UART to override cmdline or should i mount the sd on desktop and screw with files there? | 06:43 |
| kfx | I do the latter | 06:46 |
| kfx | but only because I don't know u-boot worth a damn | 06:46 |
| kfx | when I installed the huge antennae on my reform, I dropped the little brass connector, and it hit the mainboard, and somehow in that split second drained my rtc battery, so my rtc didn't work until I went to a drugstore and bought a new one | 06:47 |
| kfx | from then forward I took the 'disconnect batteries' warning a little more seriously | 06:47 |
| klardotsh | oof | 06:48 |
| klardotsh | hm. I could go the ENTIRE way back to square one maybe - is the system image that came from the factory still around anywhere? | 06:53 |
| kfx | I had a bookmark to imx8mq-mnt-reform2-system-2020-06-05.img.gz but it was back when source.mnt.re was a gitea install so it's broken now :/ | 06:59 |
| klardotsh | wait what the actual f? | 07:16 |
| klardotsh | GPU is (I think) not hardware fried? maybe? plugged in an HDMI cable. I see pre-boot gfx very, VERY briefly (looks like CPU training RAM maybe; I've seen this output in UART) | 07:17 |
| klardotsh | I'm now 10x more confused than when I started | 07:17 |
| klardotsh | when OS comes up, obvs Linux isn't initing DRM successfully, so no gfx past that point | 07:17 |
| kfx | this is a wild bug | 07:18 |
| klardotsh | just to make sure I'm not hallucinating I just Circle | 07:21 |
| klardotsh | -0'd and Circle-1'd. I see the "BIOS" on HDMI just fine | 07:21 |
| kfx | so I think the chip that is failing to train is the one that translates the mipi dsi video to something the panel can take in (edp) | 07:23 |
| kfx | which means either the chip is malfunctioning or the panel is, or something in between? | 07:24 |
| kfx | in that case it would make sense that hdmi is working, since that's a different hardware path from the gpu | 07:24 |
| klardotsh | I should clarify that HDMI does not work in OS. just in pre-boot | 07:25 |
| klardotsh | once uboot takes over, I have no graphics anywhere at all | 07:25 |
| klardotsh | I dunno if that changes your hypothesis at all, nor do I know how to continue isolating failure paths from here off the top of my head. down to try basically anything tho. | 07:26 |
| kfx | I'm out of ideas, unfortunately | 07:31 |
| klardotsh | ty for the help this far though! | 07:41 |
| kfx | if I didn't live on the wrong side of the state I'd just bring mine over and we could troubleshoot by swapping parts | 07:43 |
| josch | klardotsh: with the recent sysimage-v3, uboot should output to hdmi -- maybe you want to try that? | 07:55 |
| klardotsh | josch: that's what's currently in the SD slot | 07:57 |
| klardotsh | whatever the last green build on GL CI was as of yesterday (I know tip of master was failing, but the next commit down was passing IIRC) | 07:58 |
| klardotsh | I'm absolutely spitballing at this point but I wonder, since I know "poweroff" from OS is slightly different than Circle-0, if I managed to get the display controllers into some funkadelic state, and with eDP not able to init, kernel DRM gives up on the whole GPU and thus I also get no HDMI in OS? | 08:00 |
| klardotsh | that started sounding dumber and dumber in my head as I wrote it, remembering that I have HDMI in u-boot, but... whatever, it's a half-baked idea. | 08:01 |
| klardotsh | update: ran "reform-display-config dual" and now have full HDMI output, so it's only the eDP that's busted | 09:18 |
| + chomwitt (~chomwitt@2a02:587:dc1f:5600:72bd:2e5c:4221:2d73) | 10:27 | |
| - chomwitt (QUIT: Ping timeout: 240 seconds) (~chomwitt@2a02:587:dc1f:5600:72bd:2e5c:4221:2d73) | 11:06 | |
| minute | klardotsh: connector on the display side loose perhaps? | 11:46 |
| + chomwitt (~chomwitt@2a02:587:dc1f:5600:1db:79aa:6ee1:a01a) | 12:54 | |
| josch | interesting... the eremit lifepo4 batteries are dark purple colored | 13:13 |
| josch | i was hoping for a brighter color as in their photo https://www.eremit.de/p/18650-3-2v-2-000mah-lifepo4 but oh well, I can just get my favourite color heat shrink tubing :) | 13:20 |
| josch | again I'd like to extend the offer to the members of this channel: I have 7 barely used JGNE 18650 1800 mAh batteries that I don't need anymore -- if you pay for the shipping, then you can have them for free (i'm located in germany, so a destination somewhere in europe is probably best) | 13:21 |
| - chomwitt (QUIT: Ping timeout: 260 seconds) (~chomwitt@2a02:587:dc1f:5600:1db:79aa:6ee1:a01a) | 13:41 | |
| - mtm (QUIT: Ping timeout: 248 seconds) (~mtm@c-73-27-62-116.hsd1.fl.comcast.net) | 14:03 | |
| + Christoph_ (~Christoph@p54bf66aa.dip0.t-ipconnect.de) | 14:06 | |
| + mjw (~mark@gnu.wildebeest.org) | 14:19 | |
| josch | minute: I finally found some time to try and replicate the problem you reported last week about "reform-display-config dual" leading to an unbootable system -- with the current sysimage-v3 with linux 5.18 I don't get any output on the internal display nor on hdmi after running that command | 15:13 |
| minute | ah damn | 15:14 |
| minute | but thanks for reproducing. can we roll back the kernel version to a known good one? | 15:15 |
| minute | (not sure if i can bisect this next week) | 15:15 |
| josch | rolling back is a bit difficult because even if we put an older kernel version into the repository, apt will only always consider the highest versioned packages for installation candidates. We'd also have to somehow reproduce the old artifacts and let the CI machinery ship those. Before attempting to do all that, I'd also like to confirm that this issue really is solved by using an older kernel version. | 15:19 |
| josch | (which is difficult for me because I don't own any HDMI hardware myself and my serial is borked :D) | 15:19 |
| minute | we should swap your imx8m module asap | 15:20 |
| minute | and get you an hdmi monitor :D | 15:20 |
| josch | you think it's the module and not the board? | 15:20 |
| josch | haha | 15:20 |
| minute | i think high probability it's the module, there's nothing on the motherboard except the traces and connectors. could be soldering issue on sodimm connector manifesting late, but improbable | 15:21 |
| minute | i can sponsor a hdmi monitor or decklink usb hdmi grabber for you, whichever you prefer | 15:21 |
| josch | Lucas Stach also told me that he plans to create a single dtb for the reform, including hdmi support for the patch that he plans to submit mainline. | 15:22 |
| minute | oh wow ok | 15:22 |
| minute | good to know he has hdmi in the works!! | 15:22 |
| minute | that patch is really a big wart right now | 15:22 |
| josch | oh no, no need to sponsor anything! i only need hdmi for the reform, so me borrowing the monitor for the time I need it for testing totally works :) | 15:22 |
| minute | ok, we'll send you a monitor and imx8m module for testing if it is the serial, if not, i need to look at the motherboard | 15:23 |
| minute | if it fixes the serial i mean | 15:23 |
| josch | hm... i'd certainly appreciate a imx8m module to test whether that's the culprit | 15:24 |
| josch | but i thought you sold all you had? | 15:25 |
| minute | we recently got a few more trickling in | 15:26 |
| minute | literally trickling :D | 15:26 |
| josch | kfx: I just tried the distro_bootcmd branch again and flashed that u-boot to my sd-card. I'm not able to reproduce your findings from http://sciops.net/paste/1654891315.cb36eecb | 15:29 |
| vkoskiv | If I flash the firmware for the keyboard and LPC, which order should I do it in? | 15:43 |
| minute | vkoskiv: i would say keyboard first | 15:45 |
| vkoskiv | Cool, I'll go about doing that now. I assume anything in master | 15:45 |
| vkoskiv | master/main is safe to flash?* | 15:45 |
| minute | yeah | 15:45 |
| vkoskiv | I'll use my reform analog, the old Fujitsu machine to do the flashing. Only seems appropriate. | 15:46 |
| vkoskiv | That machine was incredible, I've had it since 2008. | 15:46 |
| josch | kfx: and the u-boot flash.bin produced by the CI also worked fine for me -- can you check again? | 15:55 |
| vkoskiv | Keeb flashed! Now to do the LPC | 16:01 |
| vkoskiv | Actually the LPC is at R3 20210925 which appears to be the current revision? | 16:03 |
| vkoskiv | Looking at git logs, there appears to be a bunch of changes since that revision was put in there. | 16:05 |
| vkoskiv | I'd personally put a git hash in place of a revision number like that, I'd like to know exactly which git commit is running on there. | 16:06 |
| + mtm (~mtm@c-73-27-62-116.hsd1.fl.comcast.net) | 16:09 | |
| minute | vkoskiv: yeah, that would be better | 16:16 |
| - GNUmoon (QUIT: Remote host closed the connection) (~GNUmoon@gateway/tor-sasl/gnumoon) | 18:10 | |
| + GNUmoon (~GNUmoon@gateway/tor-sasl/gnumoon) | 18:11 | |
| vkoskiv | I flashed the LPC and now it no longer responds. | 18:21 |
| vkoskiv | I reset it a few times and pulled the battery connectors, still not responding. | 18:22 |
| vkoskiv | Flashed it a second time, still nothing. | 18:24 |
| vkoskiv | Latest 1a9c21 commit. Any known issues with the lpc fw there? | 18:27 |
| vkoskiv | Aand resolved. The post by plattfot on the forum mentioned the lpcrc step | 18:35 |
| vkoskiv | That's the only difference I think | 18:35 |
| vkoskiv | Before I compiled it on the reform and copied the bin to another laptop to flash, and this time I compiled it on the laptop that flashed it. Potentially another difference | 18:35 |
| vkoskiv | But the LPC responds now :D | 18:35 |
| - GNUmoon (QUIT: Remote host closed the connection) (~GNUmoon@gateway/tor-sasl/gnumoon) | 19:11 | |
| + GNUmoon (~GNUmoon@gateway/tor-sasl/gnumoon) | 19:12 | |
| vkoskiv | With andwithout the lpcrc step, the resulting firmware binaries did differ. | 20:01 |
| - GNUmoon (QUIT: Remote host closed the connection) (~GNUmoon@gateway/tor-sasl/gnumoon) | 20:01 | |
| + GNUmoon (~GNUmoon@gateway/tor-sasl/gnumoon) | 20:02 | |
| vkoskiv | I want to add a refresh key for the oled menu. So I don | 22:02 |
| vkoskiv | So I don't have to hit circle+b to refresh the battery screen, for instance | 22:03 |
| josch | since the keyboard knows when the reform is plugged in because of negative mAh, can the circle+b screen not refresh automatically at a fixed interval as long as the drawn power is negative (i.e. it's charging) | 22:17 |
| josch | otherwise, as it is right now, we just get a static display that is quickly outdated after a few minutes | 22:17 |
| cinap_lenrek | http://felloff.net/usr/cinap_lenrek/lcd.c | 22:41 |
| cinap_lenrek | and the clock code | 22:44 |
| cinap_lenrek | http://felloff.net/usr/cinap_lenrek/ccm.c | 22:44 |
| minute | cinap_lenrek: great stuff!! | 22:48 |
| minute | bluerise: check this out | 22:48 |
| cinap_lenrek | i can also make you register traces | 22:49 |
| cinap_lenrek | just need to comment out these lines | 22:49 |
| cinap_lenrek | i thought uboot already has i2c infrastructure so my i2c driver isnt too interesting | 22:49 |
| cinap_lenrek | and clocks | 22:49 |
| cinap_lenrek | sorry for it being a bit hacky | 22:50 |
| - kfx (QUIT: Remote host closed the connection) (~kfx@wopr.sciops.net) | 23:17 | |
| + kfx (~kfx@wopr.sciops.net) | 23:43 | |
| bluerise | sup | 23:50 |
| cinap_lenrek | bluerise: i posted some code for the lcd setup | 23:59 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!