Jan 26, 2019

Old IRC interface for new IRC - Discord

While Matrix still gaining traction and is somewhat in flux, Slack slowly dies in a fire (hopefully!), Discord seem to be the most popular IRC of the day, especially in any kind of gaming and non-technical communities.

So it's only fitting to use same familiar IRC clients for the new thing, even though it doesn't officially support it.

Started using it initially via bitlbee and bitlbee-discord, but these don't seem to work well for that - you can't just /list or /join channels, multiple discord guilds aren't handled too well, some useful features like chat history aren't easily available, (surprisingly common) channel renames are constant pain, hard to debug, etc - and worst of all, it just kept loosing my messages, making some conversations very confusing.

So given relatively clear protocols for both, wrote my own proxy-ircd bridge eventually - https://github.com/mk-fg/reliable-discord-client-irc-daemon

Which seem to address all things that annoyed me in bitlbee-discord nicely, as obviously that was the point of implementing whole thing too.

Being a single dependency-light (only aiohttp for discord websockets) python3 script, should be much easier to tweak and test to one's liking.

Quite proud of all debug and logging stuff there in particular, where you can easily get full debug logs from code and full protocol logs of irc, websocket and http traffic at the moment of any issue and understand, find and fix is easily.

Hopefully Matrix will eventually get to be more common, as it - being more open and dev-friendly - seem to have plenty of nice GUI clients, which are probably better fit for all the new features that it has over good old IRC.

Feb 08, 2013

Headless Skype to IRC gateway part 4 - skyped bikeshed

As suspected before, ended up rewriting skyped glue daemon.

There were just way too many bad practices (from my point of view) accumulated there (incomplete list can be found in the github issue #7, as well as some PRs I've submitted), and I'm quite puzzled why the thing actually works, given quite weird socket handling going on there, but one thing should be said: it's there and it works.
As software goes, that's the most important metric by far.

But as I'm currently purely a remote worker (not sure if I qualify for "freelancer", being just a drone), and skype is being quite critical for comms in this field, just working thing that silently drops errors and messages is not good enough.

Rewritten version is a generic eventloop with non-blocking sockets and standard handle_in/handle_out low-level recv/send/buffer handlers, with handle_<event> and dispatch_<event> callbacks on higher level and explicit conn_state var.
It also features full-fledged and configurable python logging, with debug options, (at least) warnings emitted on every unexpected event and proper non-broad exception handling.

Regardless of whether the thing will be useful upstream, it should finally put a final dot into skype setup story for me, as the whole setup seem to be robust and reliable enough for my purposes now.

Unless vmiklos will find it useful enough to merge, I'll probably maintain the script in this bitlbee fork, rebasing it on top of stable upstream bitlbee.

Jan 28, 2013

Headless Skype to IRC gateway part 3 - bitlbee + skyped

As per previous entry, with mock-desktop setup of Xvfb, fluxbox, x11vnc and skype in place, the only thing left is to use skype interfaces (e.g. dbus) to hook it up with existing IRC setup and maybe insulate skype process from the rest of the system.

Last bit is even easier than usual, since all the 32-bit libs skype needs are collected in one path, so no need to allow it to scan whatever system paths. Decided to go with the usual simplistic apparmor-way here - apparmor.profile, don't see much reason to be more paranoid here.

Also, libasound, used in skype gets quite noisy log-wise about not having the actual hardware on the system, but I felt bad about supressing the whole stderr stream from skype (to not miss the crash/hang info there), so had to look up a way to /dev/null alsa-lib output.
General way seem to be having "null" module as "default" sink
pcm.!default {
  type null
ctl.!default {
  type null

(libasound can be pointed to a local config by ALSA_CONFIG_PATH env var)

That "null" module is actually a dynamically-loaded .so, but alsa prints just a single line about it being missing instead of an endless stream of complaints for missing hw, so the thing works, by accident.

Luckily, bitlbee has support for skype, thanks to vmiklos, with sane way to run bitlbee and skype setup on different hosts (as it actually is in my case) through "skyped" daemon talking to skype and bitlbee connecting to its tcp (tls-wrapped) socket.

Using skyped shipped with bitlbee (which is a bit newer than on bitlbee-skype github) somewhat worked, with no ability to reconnect to it (hangs after handling first connection), ~1/4 chance of connection from bitlbee failing, it's persistence in starting skype (even though it's completely unnecessary in my case - systemd can do it way better) and such.

It's fairly simple python script though, based on somewhat unconventional Skype4Py module, so was able to fix most annoying of these issues (code can be found in the skype-space repo).
Will try to get these merged into bitlbee as I'm not the only one having these issues, apparently (e.g. #966), but so many things seem to be broken in that code (esp. wrt socket-handling), I think some major rewrite is in order, but that might be much harder to push upstream.
One interesting quirk of skyped is that it uses TLS to protect connections (allowing full control of the skype account) between bitlbee module and the daemon, but it doesn't bothers with any authorization, making that traffic as secure as plaintext to anyone in-between.
Quite a bit worse is that it's documented that the traffic is "encrypted", which might get one to think "ok, so running that thing on vps I don't need ssh-tunnel wrapping", which is kinda sad.
Add to that the added complexity it brings, segfaults in the plugin (crashing bitlbee), unhandled errors like
Traceback (most recent call last):
  File "./skyped", line 209, in listener
  File "/usr/lib64/python2.7/ssl.py", line 381, in wrap_socket
  File "/usr/lib64/python2.7/ssl.py", line 143, in __init__
  File "/usr/lib64/python2.7/ssl.py", line 305, in do_handshake
error: [Errno 104] Connection reset by peer
...and it seem to be classic "doing it wrong" pattern.
Not that much of an issue in my case, but I guess there should at least be a big red warning for that.

Functionality-wise, pretty much all I needed is there - one-to-one chats, bookmarked channels (as irc channels!), file transfers (just set "accept all" for these) with notifications about them, user info, contact list (add/remove with allow/deny queries),

But the most important thing by far is that it works at all, saving me plenty of work to code whatever skype-control interface over irc, though I'm very tempted to rewrite "skyped" component, which is still a lot easier with bitlbee plugin on the other end.

Units and configs for the whole final setup can be found on github.

Jan 27, 2013

Headless Skype to IRC gateway part 2 - SkypeKit

Thought it should be (hardly) worth a notice that Skype (well, Microsoft now) offers a thing called SkypeKit.

To get it, one have to jump through a dozen of hoops, including long registration form, $5 "tax for your interest in out platform" and wait for indefinite amount of time for invite to the privileged circle of skype hackers.

Here's part of the blurb one have to agree to:

By registering with Skype Developer, you will have access to confidential information and documentation relating to the SkypeKit program that has not been publicly released ("Confidential Information") and you agree not to disclose, publish or disseminate the Confidential Information to any third party (including by posting on any developer forum); and to take reasonable measures to prevent the unauthorised use, disclosure, publication or dissemination of the Confidential Information.
Just WOW!
What a collossal douchebags people who came up with that must be.
I can't even begin to imagine sheer scale of idiocy that's going on in the organization to come up with such things.
And just as these things often go, here's the Pirate Bay link.
But I think I'd rather respect the right of whoever came up with that "hey, let's screw developers" policy, if only to avoid (admittedly remote) chance of creating something useful for a platform like that.

Jan 27, 2013

Skype to IRC gateway on a headless server as a systemd user session daemon

Skype is a necessary evil for me, but just for text messages, and it's quite annoying that its closed nature makes it hard to integrate it into existing IM/chat infrastructure (for which I use ERC + ZNC + bitlbee + ejabberd).

So, finally got around to pushing the thing off my laptop machine.

Despite being quite a black-box product, skype has a surprisingly useful API, allowing to do pretty much everything desktop client allows to, which is accessible via several means, one of them being dbus. Wish that API was accessible on one of their servers, but no such luck, I guess. Third-party proxies are actually available, but I don't think +1 point of trust/failure is necessary here.

Since they stopped providing amd64 binaries (and still no word of sources, of course) and all the local non-laptop machines around are amd64, additional quirk is either enabling multibuild and pulling it everything up to and including Qt and WebKit to the poor headless server or just put what skype needs there built on 32-bit machine.

Not too enthusiastic about building lots of desktop crap on atom-based mini-ITX server, decided to go with the latter option, and dependency libs turn out to be fairly lean:

% ldd /opt/skype/skype | awk '$3 {print $3}' |
        xargs ls -lH | awk '{sum+=$5} END {print sum}'

Naturally, 50M is not an issue for a reasonably modern amounts of RAM.

But, of course, skype runs on X server, so Xvfb (cousing of X, drawing to memory instead of some GPU hardware):

# cave resolve -zx1 xorg-server x11vnc fluxbox

Concrete example above is for source-based exherbo, I think binary distros like debian might package Xvfb binary separately from X (in some "xvfb" package). fluxbox is there to have easy time interacting with skype-created windows.

Note - no heavy DE stuff is needed here, and as I was installing it on a machine hosting cairo-based graphite web frontend, barely any packages are actually needed here, aside from a bunch of X protocol headers and the things specified.

So, to run Xvfb with VNC I've found a bunch of simple shell scripts, which were guaranteed to not provide a lot of things a normal desktop session does, miss stray pids, create multiple instances for all the things involved, loose output, no xdg session, etc.

In general (and incomplete) case, something like this should be done:

export DISPLAY=:0
Xvfb $DISPLAY -screen 0 800x600x16 &
x11vnc -display $DISPLAY -nopw -listen localhost &
fluxbox &
skype &

So, to not reinvent the same square wheel, decided to go with trusty systemd --user, as it's used as a system init anyway.


ExecStart=/usr/lib/systemd/systemd --user


Aside from a few quirks like hardcoding dbus socket, that already fixes a lot of XDG_* related env-stuff, proper start/stop cleanup (no process escapes from that cgroup), monitoring (state transitions for services are echoed on irc to me), logging (all output will end up in queryable journal and syslog) and such, so highly recommend not going the "simple" bash-way here.

Complimentary session units generally look like this (Xvfb.service):

ExecStart=/usr/bin/Xvfb $DISPLAY -screen 0 800x600x16

And with systemct start skype-desktop, nice (but depressingly empty) fluxbox desktop is now accessible over ssh+vnc (don't trust vnc enough to run it on non-localhost, plus should be rarely needed anyway):

% ssh -L 5900:localhost:5900 user@host &
% vncclient localhost

Getting skype to run on the target host was a bit more difficult than I've expected though - local x86 machine has -march=native in CFLAGS and core-i3 cpu, so just copying binaries/libs resulted in a predictable:

[271817.608818] traps: ld-linux.so.2[7169]
        trap invalid opcode ip:f77dad60 sp:ffb91860 error:0 in ld-linux.so.2[f77c6000+20000]

Fortunately, there're always generic-arch binary distros, so had to spin up a qemu with ubuntu livecd iso, install skype there and run the same collect-all-the-deps script.

Basically, what's needed for skype to run is it's own data/media files ("/opt/skype", "/usr/share/skype"), binary ("/usr/lib/skype", "/opt/skype/skype") and all the so's it's linked against.

There's no need to put them all in "/usr/lib" or such, aside from "ld-linux.so.2", path to which ("/lib/ld-linux.so.2") is hard-compiled into skype binary (and is honored by linker).
Should be possible to change it there, but iirc skype checked it's binary checksum as well, so might be a bit more complicated than just "sed".
LD_LIBRARY_PATH=. ./skype --resources=. is the recipie for dealing with the rest.
Skype started $DEITY-knows-where over VNC


So, to the API-to-IRC scripts then... probably in the next entry, as I get to these myself. Also following might be revised apparmor profile for such setup and maybe a script to isolate the whole thing even further into namespaces (which is interesting thing to try, but not sure how it might be useful yet with LSM already in place).

All the interesting stuff for the whole endeavor can be found in the ad-hoc repo I've created for it: https://github.com/mk-fg/skype-space

Apr 10, 2010

Auto-away for pidgin

Lately I've migrated back to pidgin from gajim through jabber.el. The thing which made it necessary was XFire support (via gfire plugin), which I've needed to communicate w/ my spring clanmates.
I'd have preferred jabber-xfire transport instead, but most projects there look abandoned and I don't really need extensive jabber-features support, so pidgin is fine with me.
The only thing that's not working there is auto-away support, so it can change my status due to inactivity.
Actually it changes the status to "away" but for no reason at all, regardless of idle time, and refuses to set it back to active even when I click it's window and options.

Well, good thing is that pidgin's mature enough to have dbus interface, so as the most problems in life, this one can be solved with python ;)

First thing to check is pidgin dbus interface and figure out how the states work there: you have to create a "state" with the appropriate message or find it among stored ones then set it as active, storing id of the previous one.
Next thing is to determine a way to get idle time.
Luckily, X keeps track of activity and I've already used xprintidle with jabber.el, so it's not a problem.
Not quite a native py solution, but it has workaround for one bug and is much more liteweight than code using py-xlib.
From there it's just an endless sleep/check loop with occasional dbus calls.
One gotcha there is that pidgin can die or be closed, so the loop has to deal with this as well.

All there is...

Get idle time:

def get_idle():
  proc = Popen(['xprintidle'], stdout=PIPE)
  idle = int(proc.stdout.read().strip()) // 1000
  return idle

Simple dbus client code:

pidgin = dbus.SessionBus().get_object(
  'im.pidgin.purple.PurpleService', '/im/pidgin/purple/PurpleObject' )
iface = dbus.Interface(pidgin, 'im.pidgin.purple.PurpleInterface')

Get initial (available) status:

st = iface.PurpleSavedstatusGetCurrent()
st_type = iface.PurpleSavedstatusGetType(st)
st_msg = iface.PurpleSavedstatusGetMessage(st)

Create away/na statuses:

st_away = iface.PurpleSavedstatusNew('', status.away)
  st_away, 'AFK (>{0}m)'.format(optz.away // 60) )
st_na = iface.PurpleSavedstatusNew('', status.xaway)
  st_na, 'AFK for quite a while (>{0}m)'.format(optz.na // 60) )

And the main loop:

while True:
  idle = get_idle()
  if idle > optz.away:
    if idle > optz.na:
      log.debug('Switched status to N/A (idle: {0})'.format(idle//60))
      log.debug('Switched status to away (idle: {0})'.format(idle//60))
    if iface.PurpleSavedstatusGetType(
        iface.PurpleSavedstatusGetCurrent() ) in (status.away, status.xaway):
      log.debug('Restored original status (idle: {0})'.format(idle//60))

Bonus of such approach is that any other checks can be easily added - fullscreen-video-status, for example, or emacs-dont-disturb status. I bet there are other plugins for this purposes, but I'd prefer few lines of clean py to some buggy .so anytime ;)

Here's the full code.

Member of The Internet Defense League