Update: New client, Xenial packages, Tox in Google Play, Toxcore fork and more!

Hello everyone! It’s been more than two months since the last update and quite a few things have happened since then, so an update is due.

Website

A new client has been added to our website — Toxygen. It’s written in python and uses Qt for its UI. Give it a try and see if you like it. You can get it for Windows and Linux from Toxygen’s Releases page. We also provide Debian and Ubuntu packages for it in our nightly package repository.

Packaging

Thanks to Encrypt’s and tux3’s efforts, we now provide packages for Ubuntu Xenial. As of writing this, the following client packages are available in nightly Xenial: Toxic, Toxygen and uTox.

New packages were added to our package repository: uTox, Ricin and Toxygen. While we did package uTox before, it was just a static binary, whereas now we have proper shared binaries made for target distributions. Ricin is currently available only in the stable repository, but should soon also be available in the nightly.

If you have been following qTox blog — and yes, qTox got its blog, we recommend you follow it if you are a qTox user — you should know that qTox has moved from distributing Linux packages through our packaging infrastructure to using openSUSE’s Open Build Service (OBS) platform, mainly as OBS allows to create packages for more Linux flavors than just Debian and Ubuntu, such as Fedora, openSUSE, CentOS and a few more. Because of this, tux3, the main qTox developer, has stopped maintaining qTox Linux packages in our package repository, but he is against removing them just yet to allow for smoother transition of users to OBS. Encrypt, the other packaging enthusiast, has said that he is willing to maintain qTox packages in our package repository in tux3’s stead, which tux3 is not against, and he will look into adding support for .rpm packages (used by Fedora, openSUSE, CentOS, etc.) to our infrastructure. Packaging is a surprisingly hard and arcane process, so if you have experience with creating proper .rpm packages in target distribution’s chroots, we could use some help — please email Encrypt.

Using Tox package repository

Just as a reminder, here are instructions on how to get our packages on Debian and Ubuntu systems.

We currently support Debian Jessie (8/stable), Debian Stretch (9/testing), Debian Sid (unstable), Ubuntu Vivid (15.04), Ubuntu Wily (15.10) and Ubuntu Xenial (16.04 LTS), i386 and amd64 architectures. To add our package repository, run the following commands, replacing TRACK with either stable or nightly and replacing RELEASE with one of jessie, stretch, sid, vivid, wily or xenial, appropriate for your Debian/Ubuntu release. Packages in the stable track are generally updated whenever a client releases a new version, and packages in the nightly track are updated once a day with the most recent development progress, if any.

echo "deb https://pkg.tox.chat/debian TRACK RELEASE" | sudo tee /etc/apt/sources.list.d/tox.list
wget -qO - https://pkg.tox.chat/debian/pkg.gpg.key | sudo apt-key add -
sudo apt-get install apt-transport-https
sudo apt-get update
# List all client packages available
grep -h 'Package: ' /var/lib/apt/lists/pkg.tox.chat* | grep -v ' lib'

There is also a special RELEASE called release available, which provides statically built versions of some of the clients. It’s not very well maintained, it’s there mostly for historic reasons, as this was the very first release name we had in our package repository and we advertised it to users. If you are on one of the supported Debian/Ubuntu releases mentioned above but you use the release release, we advise to switch to the release appropriate for your Debian/Ubuntu version.

In the future we plan on making nightly packages to be updated as soon as developers push new code for them, which might happen more frequently than once a day. We also plan on dropping Ubuntu Vivid and Ubuntu Wily support at some point, as those Ubuntu releases have reached their end of life, and adding support for Ubuntu Yakkety.

Android

Antox got published on Google Play. Previously Antox was available only though our F-Droid repo, Google Play testing and as a direct APK download, but now everyone can easily find and download it through Google Play. While we believe that it might be a bit too early to make it publicly available for everyone, given that Toxcore is not optimized for mobile yet: it uses a lot of data and keeps CPU busy, resulting in increased battery usage, which might turn people away from Tox, the Antox developer decided that it’s the perfect time to publicly release it, so here we have it. Install it, test it and report any issues to Antox issue tracker.

Shockingly enough, uTox also got published on Google Play. uTox is a native Linux application, not an Android application, made to run on Android. As such, you can expect it having UI different to what regular Android applications have, to the point that it might be not very usable. It’s surprising to see it being released on Google Play, as uTox for Android was created as a joke, just to prove that you indeed can run it on Android, it wasn’t supposed to be used by anyone. You can give it a try if you feel adventurous enough, and report bugs to uTox issue tracker, but we’d suggest waiting for uTox to be properly ported on Android as an Android application, rather than a generic Linux application.

The story goes that Antox developer didn’t want to publish Antox on Google Play until Toxcore becomes more mobile friendly, so the uTox developer published uTox to Google Play just to provoke Antox developer publishing Antox. Such mind games our developers play with each other.

Toxcore

Toxcore’s developer, irungentoo, has been very busy lately and unable to find time to work on Toxcore. Because any change to Toxcore should be first reviewed and approved by irungentoo, as he is the only one who can merge changes, Toxcore development has been stalled for some time now. To get around this, a non-hostile Toxcore fork was created by iphy, where the development is currently ongoing.

You might have noticed that the fork was created in TokTok GitHub organization. What is TokTok? TokTok initially was one of Google Summer of Code 2015 projects mentored by iphy, specifically it was the New Android Client project (GitHub repository). iphy has extended the idea since then greatly, to the point of making it a distributed storage and computing platform, but the fact that it’s still built on top of Toxcore remains unchanged. Why is the new Toxcore fork in the TokTok GitHub organization? iphy was quick on forking Toxcore, setting up automated testing and review system, opening bug reports and adding code contributions, so that before we could even get iphy to explain what the fork is about and decide whether we should support it and move it to the Tox GitHub organization, it has already became problematic to move it to Tox organization because everything would need to be re-setup again, so we just left it there, at least for now. We might move it to the Tox GitHub organization later, but for now we don’t really care where it is, what we care is improving Toxcore.

The current goal that we work towards in the forked Toxcore is to test whether Toxcore does what the Tox specification says it should be doing. Since the Tox specification was written after the Toxcore implementation was made, not the other way around, Toxcore currently has no specification-based tests. Having a specification testing framework would allow us to test Toxcore and any possible future Tox implementation against the claims made in the Tox specification, which also would allow for an easy way of extending the Tox specification and having those extensions tested. This goal also would allow us to get more familiar with Toxcore codebase, as previously only irungentoo had a good knowledge of it. After the specification-based testing will be complete and we become familiar with Toxcore codebase, we will be able to proceed with including new features into the Tox specification and implementing them.

Multidevice

Multidevice support is not yet complete, Grayhatter and several other contributors are still working on it. Some of the things that need to be done include support of synchronization of video and audio calls, file transfers and friend deletion actions.

New group chats

New group chat support is currently not being worked on by anyone. As you might have read from the previous update blog post, the new group chats are almost fully implemented in JFreegman’s Toxcore fork, the only thing needed for them to work properly is the TCP-DHT support by Toxcore. Irungentoo has said that he will work on adding TCP-DHT to Toxcore, but as he has been busy, he never got to adding TCP-DHT to Toxcore and we don’t know when he will have time to do that. Also, because the current goal of the Toxcore fork is to be fully tested against Tox specification, and both TCP-DHT and the new group chat are not (yet) part of the specification, we can’t implement them before we are done with the testing, as implementing them would make Toxcore behave in a way that is not conforming with the current Tox specification. For now we are trying to avoid modifying Toxcore behavior, until we are done with the specification-based tests. We believe that with irungentoo being less active, having specification-based tests and good understanding of Toxcore is a good investment of our time before we move to implementing other features, such as TCP-DHT for the new group chat, so don’t be upset if your favorite feature didn’t make it in, we are slowly but steadily working on it.

That’s all for now, until the next time!

Update: New group chats, multi-device, and more!

Hey everyone! It’s been a while since our last update, so we’d like to get everyone up to speed on what’s been going on in terms of Toxcore development and community happenings.

One of the most discussed topics in the Tox community over the past few months has been the release of the new group chats. Unfortunately, there is still no ETA. I finished writing the code base about 6 months ago and they’ve been available for testing for some time via my new groupchats branch. However, it’s still missing one crucial thing: TCP support.  The new group chats cannot be merged without TCP-DHT support in the core, which is currently being worked on in a private branch by the lead core developer, irungentoo. The latest word is that DHT is currently working over TCP, but he’s still having issues with bugs and failing tests. We know how eager everyone is to finally be able to try out the new group chats, and we want the community to know that work is being done and progress is being made.

In other news, Grayhatter, along with the support of Tux3, is working on another widely requested feature for Toxcore—Multi-device support. It’s still in a very early, hardly-working stage of development, but if you like the idea of testing extremely buggy code that may or may not delete your profile, you can try the Multi-device version of µTox. (Note: backup your save.tox profile first.)

Finally, we would like to offer a heartfelt thank you to LittleVulpix for stepping up to the plate and taking over toxme.io after its original owner no longer had the ability to maintain the service. Additionally, we want to thank Encrypt, who created a packaging script, and with the help of Tux3 has adjusted the script to work on our build infrastructure.

That’s it for now, but hopefully we’ll have some more announcements for you in the near future. Happy Toxing!

Tox Endorses Donald Trump for President.

The Tox project is proud to endorse Donald Trump for president of the United States.

We believe that a Trump presidency will lead to an increased amount of Tox users and that it will help the privacy movement become much stronger than it is now.

If you support privacy we ask that you vote for Donald Trump and do everything you can to make him become the next president. President Trump will help make privacy software great again.

Trump 2016 – because together we can make privacy great again.

SCaLE 14x Post-Mortem

For those who aren’t aware, The Tox Project was an exhibitor at this year’s Southern California Linux Expo, one of the largest Linux and FOSS-related conventions in the U.S. We were there to spread awareness of Tox and The Tox Project, and to get some feedback from the community regarding our work.

My experience at SCaLE started out with the excellent Friday morning keynote by the EFF’s Cory Doctorow (who later dropped by our booth to chat). Afterward, I went to check out our booth space on the expo floor and set up. It’s an interesting feeling to be setting up your booth with nothing but some stickers, a laptop, a vinyl banner, and a television, while HP and Facebook at the neighboring booths are hauling in truckloads of mainframe servers, custom-built networking switches, and the like to show off. However, despite our small-fry stature, I think that The Tox Project did pretty well for itself.

Over-all, the experience was extremely positive. Many of the expo attendees that stopped by our booth had already heard of Tox, or had used it before in some capacity. Everyone had good things to say about our efforts; the large amount of moral support we received was very encouraging. Our logo is also apparently very popular; we ran completely out of our die-cut Tox lock logo stickers, and were left with only 3 small stacks of our rectangular stickers, so I’m looking forward to seeing those Tox stickers pasted everywhere! Here are some up-close pictures of the stickers we brought:

MjzYspf

Our die-cut Tox lock logo sticker

Our die-cut Tox lock logo sticker

One thing was made very clear to me during the expo by attendees that came to the booth; Tox fills a niche in free software that very seriously needs filling. There is a need and want for something like Tox both at home and at the workplace by more people than you can count. Tox has seen some pretty rough times, and isn’t yet in the best of possible shapes, but we have incredible staying power, in large part because of the widespread desire for a drop-in Skype replacement that doesn’t abuse its end-users. Because of this staying power, we’re improving, and will always continue to do so.

In addition, a few of the things we already know were made more evident; Tox is not going to see wide adoption until we solve a few very big technical problems. Tox needs distributed solutions to the multi-device problem, the name-to-ID mapping problem (currently worked around with ToxMe), and the offline messaging problem. None of these problems are impossible to solve, they’re just very difficult to do correctly. We’ve already seen at least one proposal for a solution, but nothing that we can implement immediately; we need more manpower for research and implementation. On that note, if you (or someone you know) are a strong C programmer with experience in distributed systems, and you’re looking for a FOSS project to contribute to, drop us an email at leadership@tox.chat, and we’ll be glad to get you up to speed.

There isn’t much more for me to fill you in on, with regard to SCaLE. We spoke to very many people, and I think that we were successful at doing what we came to do.

All-in-all, SCaLE has left me refreshed. I’m glad that I’m able to start this year with a renewed vigor for The Tox Project, and I look forward to us continuing our mission to enable free and secure exchange of information via Tox.

Shout-outs to Digital Ocean, Snowdrift, Open Source Initiative, and all of the other awesome exhibitors that we got a chance to meet with at SCaLE. And a shout-out to my good friends whyt and swiss, for volunteering some of their time at our booth.

In case you didn’t catch them on our Facebook/Twitter, here are a few pictures from the event. They include Chuong Vu, an ObjcTox developer, Greg Mullen (grayhatter), the main developer of uTox, and myself (zero-one).

Setting up the booth before the expo floor opens to the conference-goers

Setting up the booth before the expo floor opens to the conference-goers

Chuong Vu (right) and myself (left) getting ready to show off some Tox clients.

Chuong Vu (right) and myself (left) getting ready to show off some Tox clients.

Greg Mullen (left) and Chuong Vu (right) explaining Tox to an expo attendee

Greg Mullen (left) and Chuong Vu (right) explaining Tox to an expo attendee

Chuong Vu (near) and Greg Mullen (far) explaining Tox to an expo attendee

Chuong Vu (near) and Greg Mullen (far) explaining Tox to an expo attendee

Having a good time at dinner. Pizza and calzones, yum!

Having a good time at dinner. Pizza and calzones, yum!

New Tox A/V API

That’s right, today is the big day! After months of work, the new API for Tox A/V has finally been merged into toxcore.

Changes:

  • A new API, similar to toxcore.
  • A better adaptive bit rate algorithm.

Nothing too surprising for the average user, but great news for developers.

The reason for this blog post is the fact that this update will break the protocol, which means you will have to update your clients. Most of our main clients are ready and if all goes well you should receive an update later today. These include: uTox, qTox, Toxic and Antidote.

Happy Toxing!

New website?

Hey guys, installgen2 here, I’ve been working a little bit on what I hope to be Tox’s new website one day. Quite a few users have complained about the current site, for reasons which are listed:

  • Proprietary JS
  • Code nobody wants to work on
  • Broken scrolling (for a lot of browsers)
  • No mobile support
  • Design inconsistent with the rest of Tox
  • Extremely heavy resource usage
  • Everything listed here
  • This issue raises some points too

So, as nobody else to my knowledge had started developing a new site, I started work on one. Unfortunately, I don’t have the time or the experience to continue development on my own, and I need help if I want to continue. I’m asking everyone with an interest in web-development and design, to help me out with the new site.

Anybody interested, please either join our #tox-dev IRC channel on Freenode, or contribute directly to the repo on GitHub. You can see my current progress at http://beta.tox.chat. Thanks!

Fuzzing The New Groupchats

With the new groupchats almost complete I thought it was time for another big testing session, and this time I decided I would take you all along for the roller coaster ride of fun. Before you read any further, I should warn that this is a technical post geared towards the programmers and developers in the crowd. If protocols and C code aren’t your thing, click here.

For those who don’t know what fuzzing is, it’s a form of random code testing. The idea is simple: feed your program random/malformed data and wait for it to crash;  gather crash info, fix the crash, and repeat (or log the crash and keep going). The purpose of this type of testing is to explore the deep dark corners of the state space in order to catch bugs that normal usage or unit testing would almost certainly miss. These types of bugs—particularly in memory unsafe languages such as C—can lead to subtle vulnerabilities that leave the door open for exploitation.

There are different levels of fuzzing ranging from dumb to smart, which explore different aspects and depths of the application protocol. An example of the dumbest form of fuzzing would be to fill a packet with completely random data and send it to the packet handler. Dumb fuzzing theoretically guarantees 100% code coverage given enough time, but in reality the universe will probably end before it ever gets past the first hash check. Conversely, smart fuzzing creates valid data and modifies/corrupts it to varying degrees. This ends up covering a smaller state space, but in practice has much greater code coverage, and is generally more useful for all but the simplest of programs.

I chose only to use two levels, where level-1 is dumb-ish and level-2 is smart-ish. Something highly complex like a compiler or a web browser might require 6 or 7 levels or even more, but the groupchat protocol is pretty simple and more straightforward by comparison. In hindsight, a 3rd level might be warranted, but each added level greatly increases the amount of work, and I’m not convinced that it would be worth the effort, as even level-1 fuzzing has satisfactory code coverage much of the time.

Before going any further, it would be helpful to understand the structure of a Tox groupchat packet:

 [  IP/UDP  |  Header 1  |  Header 2  |  App data  ]

Zooming in to the three sections we care about:

Header 1:   [  Packet ID  (1 b)  |  Chat ID hash  (4 b)  |  Sender PK (32 b)  |  nonce (24 b)  ]

Header 2:  [  Random padding (0-8 b)  |  Group packet type (1 b)  ]

App data:  [  Sender pub-key hash (4 b)  |  payload (0-? b) ]

The first header is a plain-text section comprised of integrity checks; if any of these are invalid the packet will be discarded. The second header, which is encrypted along with the rest of the packet, mitigates certain types of length-based packet analysis with padding, and tells us what higher-level function needs to be done. Due to the very simple nature of these headers and the straightforward way in which they’re handled, unit testing and manual inspection would be better suited for this section of the code. Instead, I skipped all that and focused my efforts on the app data, or more specifically, the payload.

Level-1 targets everything starting after the sender pub-key hash in the app data. The code looks like this. What it does is create a data buffer containing a valid public-key hash and a bunch of random data as the payload (this is the app data section of the packet, which is always created before adding the headers). Then the packet is sent n times to each peer in the group, where n is the number of different packet types that we want to test. If the for loops in fuzz_send_group_packet() confuse you, check out the GROUP_PACKET_TYPE and GROUP_BROADCAST_TYPE enumerators in the group_chats.h file.

A few of the packet types only have a single data field in their payload, so for them this is actually close to the highest possible level of fuzzing. An example of this would be normal text messages, where the entire payload is the message itself. On the other hand, some of the packet types have numerous fields with sensitive parsing operations tied to them, and these are the ones that require us to go a bit deeper. For example, the sync response payload looks like this:

[  num_peers  |  peer_data_1  |  peer_data_2  |  etc.  ]

What makes it complicated is the fact that peer data is itself just another series of data chunks of varied sizes (a protocol within a protocol), all of which must be treated as untrusted and potentially malicious. What happens if num_peers doesn’t reflect the actual number of peers being sent? What happens if critical parts of the peer data gets corrupted, such as the nick length? What happens if the packet is too small?

To answer these questions, we move to level-2 smart-ish fuzzing. I first need to create a valid sync response packet. There are different ways you could do this, but I chose the path of least resistance and copy-pasted most of the code for creating sync responses, adjusting things where needed. Most of that code isn’t too relevant here; the interesting part is the call to fuzz_gc_packet() at the bottom. Go ahead and take a look at what’s inside that function, as it’s the most critical part of the smart-fuzzing code, and differs quite a bit from the way level-1 randomizes packets.

Rather than randomizing every byte in the payload, bytes are randomized with a probability f, called the fuzz-factor, with use of the fuzz_this_byte() function. This is because I want to test packets with different levels of corruption. Having f constantly cycle with clock ticks means I can test payloads with a level of corruption ranging from 0 bytes to the entire thing. In addition to randomizing bytes, I sometimes also modify the data length. This tests bounds checking, although it’s only useful for packet types that don’t have a strict length requirement. For example, we would want to do this for sync responses because the sync response handler only checks the lower bound with the clause: if (length <= sizeof(uint32_t)).

Once I finished the tedious task of re-creating all the message sender functions in fuzz-form and made sure everything was working properly, it was time to try it out. I re-compiled the code with the LLVM address sanitizer enabled, and ran three concurrent instances (that is, three peers in one group, all spamming each other with fuzzed packets). I would then re-start the session every time it crashed or critically misbehaved. Some people might just settle for running it with a debugger attached, but with C there is no guarantee that invalid memory operations will cause a crash. The address sanitizer is a much more reliable tool for this purpose (though it has its own downsides that make debugging more difficult).

I ended up discovering a total of four bugs all within quick succession,  two of which were memory related (fix 1), and two of which were behaviour related (fix 2). After gaining confidence that there were no more ‘easy’ bugs, I left the three instances running overnight while I slept, and found everything running perfectly when I woke up. Eight hours sounds like a lot of time, but in the world of fuzzing it’s only about average. It’s common for people to leave their fuzzers running over the weekend, or even for weeks at a time, although it’s highly context-dependent.

With that said, I think eight hours without a crash indicates a good deal of robustness (that or my fuzzer isn’t very good; hopefully the former). While I still can’t guarantee that my code is perfectly rock solid, and will not stop testing here, I have much more confidence in it now as we get close to merging the new groupchats into Toxcore’s master branch.

If anyone has any suggestions, constructive criticism, or spots any errors, I’d love to hear about it in the comments. Happy Toxing!

Successful Indiegogo Campaign!

We are happy to announce that irungentoo’s personal Indiegogo campaign has been successfully funded!

As you may already know, irungentoo has been running a personal fundraiser, to allow himself to dedicate a month of full time work to Tox development. This fundraiser reached its $5,000 USD goal with 6 hours left on the clock. At the time of writing, a total of $5,785 has been raised by 217 funders.

The Tox Project would like to give a big “Thank You!” to all that donated, as well as to all of Tox’s supporters for making this happen, and helping to raise even more money than irungentoo’s original goal.

irungentoo would like to say to everyone:

Thank you, everyone, for making this fundraiser a success. If you bought a perk please send me the related stuff, it will be put into the repository when I collect most of them. If you donated bitcoin in an amount equal or greater to the value of one of the perks and want to receive that perk, just email me at irungentoo@gmail.com with proof (e.g. a message signed with your bitcoin address key).

Unless something happens (computer stops working, etc.), I will be starting my full time work on toxcore on September 8. I will start by writing toxcore documentation, which will take a long time but is important and might invite more people to contribute to toxcore and make sure that Tox does not die if I get hit by a bus. Thank you for making sure that Tox will become even stronger.

Sincerely,
irungentoo

We’ll be sitting down with irungentoo very soon to develop a roadmap for the toxcore library. Once it has been developed, we’ll publish it, and you’ll be able to track development progress as it happens.

Thanks again,

The Tox Project

Sound design!

As I’m sure you are all aware, Tox has grown tremendously in terms of UI and usability, yet client sounds are still somewhat lacking.

We are looking for people to help out with creating sounds for clients (ringtones, notification sounds, etc). No programming knowledge is required. Just join us in #tox-dev on Freenode (here is webchat for those of you without an IRC client).

Thanks!

Tox Dev Talks – #3

(Tox Dev Talks is a series of weekly meetings that bring the Tox developer community together to share ideas, progress, and miscellaneous chatter. They take place Saturdays at 16:00 UTC.)

In this week’s meeting, as usual, we discussed a variety of topics, which have been summarized below.

(Edited 2015/08/30: a few factual corrections were made. Apologies for letting them slip past editing -ak)

  • Binaries
    • Package signing
      • Jenkins will handle it
      • Possibility of making Jenkins push notify on changes
  • Policy
    • #tox-dev
      • We discussed how much off-topic discussion and other noise we are willing to allow on #tox-dev
      • Possibly enabling colors for improved build info
      • nurupo expressed concern of color misuse
    • Social media
      • We reminded the managers of each Tox-related account of their responsibilities
      • Blog
        • Social media will now link to EVERY blog post
        • GSoC mentors should come up with a GSoC blog post
  • Our website
    • installgen2 is working on redesigning tox.chat from scratch, this will take 3-7 days to complete
  • ToxAV
    • Mannol has said there will be more commits this (2015-08-29) weekend
  • New groupchats
    • JFreegman is still waiting on irungentoo to implement the difficult codebase changes
  • Core and client updates
    • qTox
      • Zetok will be tagging qTox issues
    • uTox
      • in-progress UI redesign
      • New groupchats once ToxAV gets merged, and the refactor in that branch gets merged
  • Other developments
    • nurupo is working on fixing bugs and refactoring code in code base of GSoC project he mentored
    • codedust is developing https://github.com/codedust/WebTox – a web based tox client