nikclayton

Mastodon: https://mastodon.social/@nikclayton

Edit to add this set of links to the posts in the series

That was an interesting 24 hours or so.

First of all, if you're one of many people who boosted or commented on the original post, or left feedback appreciating any of the open source work I have done, thank you.

After the project's financial admins shutdown discussion I spent some time reassessing my concerns, making a gut-check they seemed valid. It's still a risk to bring these sorts of issues to a public forum, and I'm relieved so many of the replies and commentary I've seen elsewhere agrees that yes, this is something to be concerned about.

I said I'd collate responses to issues I'd seen raised and reply in a single post, so that's this. I'm aware it's not an ideal format, but neither is fitting each reply in to multiple 500-character posts and hoping they federate properly so everyone sees the full discussion either.

If I've missed something below please feel free to send me a DM and I'll follow up in the next update (I'm pretty certain there'll be one, if only to announce a new git repository...)

Thank you for posting this. It is indeed concerning.

Whatever you do in the future, please consider not creating your own legal entity but instead joining one of the existing (well respected) fiscal host organizations like Software Freedom Conservancy, Software in the Public Interest, etc. They can handle all of the assets, legal paperwork, liability, etc. - https://mastodon.km6g.us/@kevin/110963590931224115

Yes, that's a very important point, thanks for raising it.

To organise my thoughts about this I started framing them as pitch for a new association. The early draft is pitch.md.

There's a great deal of overlap between what I describe there and the work other organisations do, and I would prefer to work with an existing organisation than contribute to further fragmentation.

I have already contacted

After some research those seemed to be the most appropriate three, but I may have missed some, so if you, or anyone else, can recommend other organisations that would be a good fit I'd be happy to hear it.

so, to summarize, you found a payment to a contractor suspicious because you couldn't find any code from the work, and then you talked to the contractor and found the code, so your suspicions were in fact unfounded.

And also you think it's a conflict of interest for the person who does the accounting for the org to also be on the payroll for he org... like an accountant - https://social.coop/@datatitian/110964853363902591

I see several other people have already replied to you correcting the inaccurate summary in your first paragraph.

Re the second, an accountant does not generally have sole approval authority over expenses. Consider this hypothetical. A financial admin drawing significant funds from the project receives a funding proposal from someone else. Approving this proposal would reduce the amount of money available for them, so they decide to reject the proposal.

There's the conflict.

I am not certain that the first two counts are suspicious. Payment was made for work on three issues. It sounds like the person was legit, they just didn't finish the work after being paid?

The second is just silly. Project Management is a job title and job description. It is explicitly what they mean by “someone unfamiliar with your project” will recognize.

Invoices don't list specifics. There should be internal docs for that. Those docs often aren't public.

If the project (hypothetically) hired me, I'd appear on the invoice as “Policy & Regulatory Analysis”. You would NOT list the different policy issues involved on the invoice. That would be documented internally for many reasons. What the accountants need to know is “what was this money used for?”

I do not work on IT regs, so I cannot speak to specifics, but I do think that it's better to ask an attorney or accountant to look first before raising suspicions. - https://med-mastodon.com/@UncivilServant/110964976957986119

It's not “didn't finish the work”, it's “didn't do the work”. The payee may have worked on the issues in private. But if they don't push the work somewhere public, create PRs from it, link to it, etc, then it is indistinguishable from not doing the work.

As I wrote in the initial post, there was no public indication any work had been done on these issues.

In your “Policy & Regulatory Analysis” example, as an open source project, I would expect to see, at minimum, a link to the work you produced as part of that analysis. That would be “sufficient detail to make it clear what the collective is paying for”, per the OpenCollective guidelines.

I would also expect any agreements the project makes with third parties to clearly set out what the expected deliverables are.

This works even for problems where the shape of the solution is not known beforehand.

For example a bad agreement would be “Solve problem X”. After a month the payee comes back and says “Sorry, I tried lots of things, I couldn't solve it, I've got nothing to show for it, here's my invoice.”

A better agreement would be “Solve problem X. If you can't solve X, deliver detailed descriptions of the approaches you took and why they don't work, so future attempts to solve this problem can build on what you learned.”

That's a lot of book keeping and managing overhead tbh. [...]

My greatest fear of launching monetary-focused foss products is this management overhead which compared to the rest of the industry just feels so unworthy. The monetary incentives just don't add up so I can definitely see how easy it'd be to get lost here. - https://fosstodon.org/@wraptile/110965923454931957

Yes, I agree. Introducing expenses, payments, grants, etc in to a project can add a lot of overhead.

Projects should not do this unless they have the necessary people, time, and expertise to do this properly.

In other words, don't adopt processes you're not staffed to meet (a variant of a rule I described in Rules for a healthy on-call rotation as “Don't set SLOs you are not staffed to meet”).

You’re expecting the financial admins to do their work for free. If anything, you’re making yourself look bad here, and not the Open Collective. - https://eupolicy.social/@whvholst/110965996322449276

I'm definitely not expecting someone to work for free. If the only way someone can contribute to an open source project is to be paid, and the project has funds to pay them that's absolutely fine.

That does not absolve them of the responsibility to do things properly, and with accountability.

Finally, this thread:

I've said this in the past, but maybe today it's more important than it's been in a very long time to tell you.

Thank you everyone who's been supporting #Tusky over the years on open collective. You've been supporting my disabled ass through so much, and I always knew I had a little bit of money at the end of the month, even when I was the most sick. ($100 when sick, $175 when able to work)

Me and Conny worked out this agreement back sometime after I started the open collective for the project (early 2019?). I've mostly been providing support, and project managment at some points. Organizing people and texts. A lot of invisible work. . I'll always be eternally grateful for everything you've done for us and me. - https://rage.love/@maloki/110969436075202543

If this shit turns out to be a techbro devaluing the invaluable (and largely taken on by women) glue work Yet Again I will yeet my chair through my office window. - https://comicscamp.club/@nebulos/110969752596667389 (direct link broken at request of the original poster)

i mean it partially is. - https://rage.love/@maloki/110969756892267642

It really partially isn't.

I was careful in my original post not to name anyone directly, I was expecting a collective response from the Tusky project. Since Maloki has come forward in public I won't anonymise comments from her in the future. I will continue to redact the names of anyone else in the project unless they also respond separately from the project.

Having dealt with that, there's quite a lot to unpack here.

First, @maloki@rage.love is the financial admin quoted in the original post, who initially insisted the discussion be shut down, and as is apparent from these posts, also the recipient of a substantial fraction of the project's funds.

That conflict of interest should be noted.

Second, an uncharitable reading of this thread — and while I continue to assume these issues were mistakes and not a deliberate attempt to hide wrongdoing, the needle on that dial is starting to wobble — could see this as an attempt to derail this conversation away from the concerns I raised, DARVOing the discussion.

Let's not do that, please.

Third, I think a reasonable reading of “$100 when sick, $175 when able to work” is the project was paying you during months when you did no work.

That seems to be supported by invoices like #30003, “Sick-pay June-December”.

If that's a mis-read of that line I'm happy to be corrected.

If that's not a mis-read then I have a host of other questions, starting with, was the project effectively paying you a salary as an employee?

I believe you're in the UK, so even if you think you were an independent contractor, IR35 probably applies (not an accountant, but I used to run a contractor business in the UK, so I have experience with this) and this probably should have been treated as employment income.

  • Was that legal?
  • Was the project paying you at least minimum wage (GBP 10.42 an hour)
  • Was the project paying appropriate taxes and employee insurance contributions?
  • Does the project have an employee-like arrangement with anyone else?
  • Why wasn't the process for using OpenCollective to pay employees (Employment & Benefits – Open Collective Foundation) followed?

Fourth, your comment “sometime after I started the open collective for the project” prompted me to go and look at those first invoices, at Tusky · Expenses – Open Collective

Of the eight invoices on that page. two are for Tusky project expenses (domain name, stickers), one is payment to a translator, for a total of USD 129.53. The rest (USD 960) are payments to you.

An uncharitable person could think the project did not actually need the funds, and that you started the project's OpenCollective to provide a front that would allow you to collect money hoping that it wouldn't be noticed.

A charitable person could assume that is not the case, but still agree that the optics of this are terrible, especially after you repeatedly tried to shut down any discussion of the project's invoices.

Fifth, “I've mostly been providing support, and project managment at some points. Organizing people and texts. A lot of invisible work”.

Undervaluing people's contributions is an ongoing issue, and a bit of a hot-button topic. If you're new to this I recommend Being Glue — No Idea Blog.

So let me make a couple of hopefully non-controversial statements:

  1. A quid pro quo for taking money from the project for “invisible” work is it's on you to make sure the work becomes visible.

  2. If you can't describe what you're doing for the project, how is the project going to replace you when you step down?

I know what that work looks like (I'm aware of the “invisible” / “looks like” irony), I've spent years in previous roles doing project management work, I've had the good fortune to be mentored by some truly excellent project managers, and I've worked to get people successfully promoted by ensuring that the impact of their glue work was recognised.

And I've had the opportunity to learn from some epic screw ups of my own.

So here's a short list of things you could have held yourself accountable for doing:

  • Curating the list of open project issues (as in #3240 for example)
    • Following up on incomplete bug reports
    • Checking if new releases solved the reporter's problem
    • Making sure that bug reports were not being ignored
    • Applying consistent labels to make them easy to manage
  • Improving the onboarding experience for new contributors
  • Planning a release calendar, or any form of release scheduling
  • Writing the release notes for each release
  • Proactively adding new contributors to the GitHub repo
  • Finalising a complete expense policy
  • Finalising a handover document to prepare for your departure (document promised since early 2023)
  • Following up on stalled PRs
    • Had the submitter run out of time?
    • Did they not understand the review feedback?
    • Were they waiting for review and the project had slipped up?
  • Creating GitHub issues to track problems reported by users to the @tusky account
  • Communicating the state of ongoing activities to the rest of the project contributors
  • Tracking requests for comments on proposal documents
  • Liaising between the Tusky project and other Fediverse projects
    • I'm aware of your overlap with the GoToSocial project. You seemed actively hostile to any communication with the folks at Mastodon GmbH.
  • Outreach to sites comparing different Mastodon clients, ensuring their info about Tusky was accurate
  • Keeping the OpenCollective page up to date with project news
    • I asked for this multiple times — it could have been as simple as copying the release notes in to an announcement after each release.
  • Regular check-ins with each contributor to the project; how are they doing, do they have the support they need, do they have any issues that need addressing, etc
  • Responding to inaccurate or mistaken reviews on Google Play

Not all of these things were being done by the project, but the ones that were were all being done by other people. Framing your contributions as non-specific “providing support, and project managment at some points. Organizing people and texts. A lot of invisible work.” is a very effective way for other people to get the impression that you must have been involved in these things, denying the credit to the people who were actually doing the work.

I'm not for the moment suggesting you have to do all of these things. But doing one or two of them effectively would have reduced the burden on other project contributors who weren't, it seems, being paid a salary.

I'll also highlight the distinction between being accountable for something and being responsible for something.

  • Accountable – I will make sure this gets done, I will find the right people to do it
  • Responsible – I will be the person actually doing the work

You may not have been the person who could be responsible for a piece of work, but you could have been the person who was accountable for a piece of work.

The concrete things I think you did in the 8 months I was involved (often after you were prompted to do them) were:

  • Creating chat rooms
  • Organising and chairing the first team meeting (after multiple people called for one to happen)
    • Subsequent team meetings were completely handled by [redacted], who scheduled them, managed the agenda, and chaired the meetings
    • That was helpful, the first meeting was run well, and [redacted] is continuing to do an excellent job
  • When @connyduck's private message stepping down from the project was leaked I suggested the project post an announcement, and posted a draft for discussion. You wrote your own announcement and posted that, also updating OpenCollective for the first time in more than four years.
  • Occasional comments on the chat groups or on discussion documents, no different from any other contributor.
  • An attempt to form a more formal support group with [redacted] that, to the best of my knowledge, didn't achieve anything

You may also have been doing some customer support from the @tusky account at some point before early 2023. It's difficult to tell because we didn't indicate who was making each post from the account. I'm confident you did very little, or none of that, from the time that I was also posting from the account, because I would have seen it.

Sixth, most of this is the project's problem. If the agreement (can we see it?) did not state what you were expected to do beyond vague “project management” then it becomes a lot more difficult for the project to say “hey, please can you do some of the things you're being paid to do”.

But recognising that issue, and working to correct it is a responsibility I would expect you to share.

And trying to shut down any discussion is still a terrible look.

Edit to add this set of links to the posts in the series

I discovered severe lapses in how the Tusky project's donations (received via OpenCollective) were being handled. When I reported those to the project's private “Tusky Contributors” Matrix channel the financial admins tone policed the feedback, refused to engage with the concerns I and others raised, and demanded the discussion be stopped.

That's an ethical boundary I'm not going to cross, and I'm not interested in collaborating with people who have crossed that boundary and are not prepared to walk back.

Therefore I am stepping back from the Tusky project with immediate effect, and I recommend anyone who is currently donating to the project via OpenCollective reads the following, and makes their own decision on whether or not to continue financially supporting the Tusky project.

I do not have a conflict of interest; I have not taken money from the Tusky project in the past, I do not have any open expenses with the project now, and none of my income is remotely related to the Tusky project or any competing projects.

I'll be reading any replies with questions, collating the questions directed at me, and then writing a single response addressing all of them (or one per day if the questions run over multiple days).

Why were you looking in to this?

@connyduck's recent departure from the Tusky project exacerbated an ongoing issue with the ownership of project assets (domain names, website hosting, Google Play publishing account, etc). They were owned by him, sometimes entangled with his personal accounts, and would need to be transferred somewhere new so the project could continue, and to allow him to completely disengage from the project.

After discussions with other contributors I started researching options for creating a separate legal entity to own these assets on behalf of the project.

The research showed that whatever specific form the entity took it may involve some people taking on legal liability for the project's finances.

To determine what the impact of that might be I started looking into the Tusky project's OpenCollective history to see what sort of things the project was paying for, how much was being spent, what oversight there was, and so on. This would let me recommend alternatives that matched how the project was currently doing things, and highlight anything we would need to change.

What were the lapses?

OpenCollective's policies on how projects should record expenses are very clear (Invoice and Reimbursement Examples – Open Source Collective, emphasis mine):

Invoices need a clear description of services rendered and provide sufficient detail to make it clear what the collective is paying for. This is because this description may be checked by our accountant for compliance. That means that someone with no knowledge of your project, and only a limited knowledge of open source, should be able to understand what was done from the description.

Many Tusky project invoices do not do this. And in some cases they show payments for work that was demonstrably not done.

The first invoice I noticed this on was Invoice #125597, which is for:

Work on 'bookmark tab' feature (#2368); Work on 'copy hashtags into reply' (#3013); Work on 'positional substitution format' (#3297)

That's from February this year, it's now late August.

There is no publicly available evidence of any work on those issues.

  • The payee's GitHub repository (GitHub – SomaRasu/Tusky) had no indications of any work done:
    • No branches for the work
    • No commits relevant to the work
    • No PRs proposed with that work (from either this repository, or any other)
  • The three mentioned issues were still open, #2368, #3013, #3297
    • There were no comments on those issues from anyone
    • There was no indication any work was underway to resolve those issues
  • The payee had not asked for help with these issues in any Tusky public or private Matrix channels I'm a member of, or public channels I searched through

Note: After initially raising this issue I then spent approximately two hours on 2023-08-23 and 2023-08-24 mentoring the payee to fix issue #2368, and submit a PR. They did, which I reviewed, approved, and merged. The other two issues remain open at the time of writing.

Important: I do not think any of this is the payee's fault, and have told them this. I believe the project did not provide them with sufficient support, and when it became clear they could not complete the work the project could have paused the engagement, or asked one of the other contributors to assist.

Further review showed other invoices that did not “provide sufficient detail to make it clear what the collective is paying for” and allow “someone with no knowledge of your project [...] to understand what was done from the description.”.

For example, Invoice #136617. You will find many more examples at Tusky · Expenses – Open Collective.

Why is this a problem?

Obviously, there is the violation of the OpenCollective policies.

For me, it is also a violation of the project's ethical duty to responsibly and transparently handle donations and payments.

Transparency in how people are selected to receive funds, and how much to pay, is especially important in an industry rife with bias, and where women are regularly underpaid for their work.

And due to the nature of the Fediverse, Tusky attracts funding from people who have been marginalised both historically and today. I think our duty of care to be responsible and ethical stewards of the funds is even greater because of this.

Finally, a large red flag I subsequently discovered is that one of the members of the financial team managing the OpenCollective donations is a regular recipient of funds from the project. Of the USD 3,908.56 paid out by the project in the last year (2022-08-25 to 2023-08-24) (Tusky · Expenses – Open Collective) they have received USD 1,830.00 (47% of the total).

I am not suggesting those invoices are illegitimate, or the work they paid for was not done.

But I do think good financial stewardship, as well as straightforward common sense, would have recognised the obvious potential for conflict of interest, and ensured that anyone who was so dependent on the project's funds would not have any say in controlling those funds.

Raising the issue

I first flagged Invoice #125597 as a concern in a message to the private and invitation only “Tusky Contributors” Matrix channel on 2023-08-21 (Monday).

The initial response from other Tusky core contributors noted they couldn't find any information about the invoices in question, and agreeing, on the face of it, the work described in these invoices was not done.

The Tusky financial admins then responded, noted that this was for work done under a separate contract, tried to derail the conversation by raising their own expenses, and attempted to blame me, claiming that since some of my first opensource contributions to the project overlapped with work that this person under contract was supposed to be doing, my contributions caused a problem with their work

As a first time contributor to the project at the time I was unaware of this, because the work they were supposed to be doing was completely undocumented and there was no public indication that were working in the same area.

This raised more questions. In particular, I could find no traceable work from the person under contract in the Tusky project; no commits, no PRs, no issues, no comments on issues, no questions asked in the public or private channels that I'm a member of, etc.

I asked what work was actually done, flagged the risk that without more context this could appear to be fraud, and was very clear I thought this was a straightforward mistake, writing (emphasis in the original):

To be super clear — I'm 100% not saying it is fraudulent. This could be as simple as “Some work was intended to be done, it wasn't, other work was done instead, and the expense description was not updated”.

But we should be able to explain what that work was, and we should have processes in place so that this does not happen again.

The admins:

  • Told me to stop asking questions
  • Said that being required to show proof of work would call their own invoices in to question
  • Insisted this was an issue that only the admins needed to know about

Other contributors continued to question this.

In response, the admins:

  • Swore, repeatedly
  • Insisted the contracts and general process would not be available to contributors

The discussion deteriorated from there, the admins refused to answer direct but reasonable questions, insisted that the conversation should stop, and described questions as “entitled”.

At this time my specific unanswered questions are:

  • Does the project have agreements in place with anyone else?
    • If so, with who, and what are the agreements?
  • Does the project currently plan to have agreements in place with anyone else?
    • If so, with who, and what are the agreements?
  • If it is technically possible, will the project amend the notes on previous OpenCollective invoices so they meet the letter and spirit of the OpenCollective policy?
  • Will the admins put in place a formal expenses and grants policy, with a public review period before adoption?
  • Is the project paying contributors an equal equivalent hourly rate?
    • If yes, what is the rate?
    • If no, how is the effective hourly rate determined for a contributor?

If the admins had:

  • Acknowledged the concerns
  • Agreed to investigate what happened
  • Agreed to put controls in place to prevent it happening in the future
  • Agreed to not processing any further expense claims until those controls are in place

then I wouldn't be writing this now. They didn't, so I am.

Could this be an innocent mistake?

Yes. I think the Tusky financial admins have made a number of serious, but correctable mistakes. I do not think anyone set out to deliberately defraud the project, or its donors, and I was clear about that during the discussion (see earlier).

However, instead of correcting those mistakes when the issue was raised they doubled down on them.

Their position is secret agreements in place to pay out funds from the Tusky OpenCollective are OK, and project contributors and donors should not be able to discuss these agreements before, during, or after the period they run for.

Some admin quotes from the discussion:

Can You stop? [...] This is an admin issue. The admins had agreements in place with people involved.

[...]

these conversations aren't going to be for and open to the contributors.

[...]

it's not a question that necessarily should be posed in the contributors channel

[...]

the contracts and agreements we make with people is not for public consumption, and I don't believe they have to be, which is why I was very thorns out on this question. You don't need to know any details of the agreement whatsoever besides “this was a past agreement and it got paid for”.

They also said:

we've worked on improving the process

But, when asked, refused to explain what those improvements were.

As noted earlier, other project contributors also expressed their concerns in the discussion, with comments including:

to me it also looks very odd. [...] in this case it does really look like payment for work that wasn't done. I think big point of OC is transparency and this does not look good for us. [...] I agree that we should be transparent and should have a record of agreements

[...]

That said, if it doesn't comply with OC, it can't be allowed to continue to happen like that. Especially not if Tusky gets a legal entity. With likely a legally liable treasurer. Also, especially not if the original core decision makers are also stepping down.

Who are you? Why should I care about your opinions?

I'm Nik, I've been contributing to open source projects since the early 1990s. I've run large open source events with a large financial outlay (I was one of the co-founders of BSDCon Europe back in 2001). I've started open source projects, joined open source projects, been given stewardship of existing projects as maintainers move on, and handed stewardship of projects I've started over to other people.

I've been contributing to the Tusky project since December 2022. My first contribution improved the FAQ (#16). Since then I have contributed more than 150 PRs to the project. If you've been affected by bugs like:

  • The swipe sensitivity between tabs being too high (fixed: #3148)
  • Losing your place when you press “Load more” (fixed: #3000)
  • Missing notifications (fixed: #3700 (and many others))
  • Images getting “stuck” when trying to zoom or swipe between them (fixed: #3894)

then I'm the one who fixed them.

I've also improved the app's accessibility including #3003, #3121, #3272, and #3248.

As @connyduck started reducing his involvement with the project I took on the responsibility of producing new releases, managing the releases for Tusky versions 22.0 and 23.0. That includes running the release process, writing the release notes, managing the beta programme, responding to user bug reports, and so on.

At the time of writing the Tusky OpenCollective site describes me as a “Core contributor”. Per the Open Collective documentation “Core contributors show up in the Team section of your page and can create events, but can't change settings or approve expenses.“, so this gives me no special access to project accounts or contracts.

For the last several months all (or very nearly all) of the posts or replies from the @tusky@mastodon.social account have been from me. And a cursory review of PRs over the last few months will show the majority of them are reviewed and merged by me as well.

In short; I know open source. I get work done. I know what good project governance looks like.

Why are you not naming names?

I think this is a collective failure of the project's financial admins, not any single individual, so the names of the individuals involved is not directly relevant.

I do not want to encourage a typical Internet pile-on of them, and any questions you have for them should be directed to the Tusky project's account, @tusky@mastodon.social (I am no longer monitoring or posting from that account).

By necessity a handful of the links above clearly show e.g., who submitted an expense, or was responsible for an approval. Again, please do not pile-on to those individuals, but send your questions about the project to @tusky@mastodon.social.

Do you have any more proof?

Yes. I have the chat logs of the “Tusky Contributors” Matrix channel that I am a member of, and where this issue was raised and dismissed.

I also have copies of the meeting minutes from the handful of Tusky contributor online meetings that have happened this year.

I will release the unredacted logs and/or meeting minutes if the the project's financial team agree, or those people make public statements that contradict what they said in private.

What's next?

I doubt the Tusky project financial admins will change direction. If they publicly commit and follow through on a transparent financial policy that meets the OpenCollective guidelines then I am prepared to rejoin the project and continue contributing. But if I expected that to happen I would not have written this.

If you are a financial contributor to the project I hope this has given you enough information to make an informed decision about where you send your money in the future.

Some Mastodon server owners are forming member-led cooperatives to host their online communities. I have been following CoSocial Community Cooperative as one example Social.Coop is another.

I think there may be space for a Mastodon app run along the same seven cooperative principles.

  1. Voluntary and open membership
  2. Democratic member control
  3. Member economic participation
  4. Autonomy and independence
  5. Education, Training, and Information
  6. Cooperation among Cooperatives
  7. Concern for Community

This incident has prompted me to investigate starting a project to do just that. More information soon.

Some people will cheerfully tell you that Mastodon is a different kind of social network because it doesn't have an algorithm.

Other people will angrily rail against this, insisting that “sorted in chronological order” is an algorithm, it's the only algorithm, it can't be changed, therefore Mastodon is bad because it doesn't provide any user controls.

Every now and then I get sufficiently annoyed by dumb takes about this to do a write-up. This is one of those times.

xkcd: Someone is wrong on the Internet

xkcd: Duty Calls

Go on then, tell me how people are wrong on the Internet

Both the positions above are wrong because they're lumping two completely different algorithms under the word “algorithm”.

Those algorithms are:

  1. How does Mastodon select the posts that will appear in your home timeline?
  2. How are the selected posts displayed?

The “Mastodon has no algorithm” crowd are actually saying “Mastodon has an algorithm, but it's fully controllable by you, nothing will show a post in your timeline that you did not explicitly ask for”.

As best as I can tell the “Mastodon absolutely has an algorithm” crowd don't care about algorithm #1 at all, and are focused on algorithm #2, insisting that “chronological ordering” is an algorithm, so that they have something to be angry online about.

There's a third algorithm too, How is an individual post on the home timeline displayed?, but people hardly ever talk about that one.

Selecting posts for your home timeline

This is the algorithm that actually matters.

You don't need to read the Mastodon code to understand how this works, you can deduce it from observation.

First:

  • Include all the top-level posts from people you follow
  • Include all the replies from people you follow
  • Include all the boosts from people you follow
  • Include all the posts containing hashtags you follow

Then:

  • Remove replies if the user has requested that
  • Remove boosts if the user has requested that
  • Remove matching boosts if the user has enabled the “Group boosts in timelines” settings
  • Remove posts from users who are actively muted
  • Remove posts from users who are blocked
  • Remove posts that match the user's filters to hide

That's at least two different ways the user can control what is initially chosen for their home timeline (users and hashtags) and at least six different ways the user can then control whether those posts appear on their timeline.

Displaying posts for your home timeline

The sort algorithm barely matters. It's “sort chronologically, newest first”. This is not modifiable by the user in Mastodon.

Mastodon does provide additional controls for the algorithm that determines how each post within the timeline can be displayed.

You can choose:

  • Whether to show media in general
  • Whether to show media that the poster marked as “sensitive”
  • Whether to automatically expand posts that have a content warning

And for posts that have been filtered at the “warn” level the user can, on a post by post basis, choose to view that post.

So, again, there's an algorithm for how each post is displayed, and the user can make choices that affect how that algorithm behaves.

Caveats

A few caveats about the above because this is the Internet and people will nit-pick to death.

First, it's describing the current observable behaviour of the Mastodon server and default web client.

Second, other clients (both web, and apps) for Mastodon servers exist, and some of those provide further controls on the presentation of your timeline.

For example, the Phanpy web client can

  • Display boosts separately to the timeline, in a carousel
  • Shows conversations in a threaded view grouped together in one block in the timeline

See Phanpy subtle UI implementations for more on that, including screenshots.

Third, Mastodon-like servers exist; federating with Mastodon servers, supporting the same basic API, but providing their own additional controls over what posts appear in your home timeline.

For example, the glitch-soc server implemented fine-grained post filters before Mastodon did. So for a while, users on a glitch-soc server had more control over their timeline than users on a Mastodon server.

In summary

  • Mastodon, and Mastodon-like servers, absolutely have an algorithm they use for selecting the items that make up your home timeline
  • The specific client you're using to access your Mastodon server might also provide its own algorithms for displaying the timeline, which you can adjust
  • Unlike other social media services (Twitter, Bluesky, Threads, etc), a Mastodon server is extremely transparent about the algorithm selecting the posts for your home timeline, and gives you full control of the parameters used in those decisions.
  • Ignore anyone arguing “Mastodon does not have an algorithm”, they lack critical thinking skills
  • Ignore anyone arguing “chronological order is an algorithm”. When it comes to social media “How are the posts you see chosen?” is a much more important question than “How are those posts ordered?”. If that's what they're focusing on then they're probably just looking for a reason to be angry about something online.
  • “I would like Mastodon to provide a new option X, which would affect how posts are chosen for my home timeline” is an interesting discussion to have. But that would require careful thought, and most people arguing online don't have time for that.

Prompted by the questions in https://hhmx.de/@nick/16376, the answer's too long to fit in a post.

Doing some monitoring how a original #Mastodon server responds, i never (!) saw the described link header! The response may include no link header or a header like this: Link: ; rel=“next”, ; rel=“prev”

You should get a Link header from any API response that returns a page of data, like https://docs.joinmastodon.org/methods/notifications/#get.

[Almost; in testing this I just discovered an edge case, which I've reported as a bug at https://github.com/mastodon/mastodon/issues/25495]

An API response for a single item, like https://docs.joinmastodon.org/methods/notifications/#get-one won't include the Link header.

I just tested this with https://restfox.dev against my mastodon.social account (https://github.com/tuskyapp/Tusky/pull/3610/files?short_path=00c65c2#diff-00c65c2349f395f9f9b78b0fe84d3638b78f3ef4abf98294364b5c4d0ef32473 has details on how to do this).

On my research i saw something in the #Tusky source, that looks like Tusky as clients seems to generate a link header? If so, why?

https://github.com/tuskyapp/Tusky/blob/develop/app/src/main/java/com/keylesspalace/tusky/components/notifications/NotificationsPagingSource.kt#L107-L178

That function needs to return a page of results as an HTTP Response. The code that calls that function unpacks the Response and parses the contents of the Link header, so the header must be present.

To figure out what page of results to return one of the parameters to getInitialPage is the ID of the item (in this case, a notification) that should be at the start of the page. This is the params.key value.

The Mastodon API lets you make requests like “Get me the page immediately after ID X” or “Get me the page immediately before ID X”.

But it does not let you make a request “Get me the page that includes key X”.

So the code has to do that itself, starting around line 134. It does this by making two API calls. One to retrieve the notification with ID X, and one to retrieve the notifications immediately after that.

If those calls succeed the code has two values to operate on; the notification, and the page of notifications immediately after it.

Remember that this function needs to return a single page of notifications. So it creates a fake page which contains both values. This fake page needs a Link header (because the code that calls this code expects a Link header).

So that's why it constructs one in this case.

This code is only called to get the first (or initial) page to show the user. After that the pages above and below are loaded with single calls to the Mastodon API, and return the Link header without any alterations.

Besides of details, if the header should be named “Link:” or “link:“,

HTTP headers are case-insensitive (https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers) so you can use either.

Tusky 22 now sometimes reports an array index out of range (-1) or similar (taking this from my memory, for exact message, i have to check it again).

That would be a bug, a detailed report with reproduction steps would be appreciated.

During development i also saw Tusky sending requests for notifications in an endless loop... wondered why.

I suspect the min_id / max_id handling in your server is not correct.

And... could someone clarify the difference between sinceid and minid? If Mastodon will handle this differently?

sinceid = String. Return results newer than this ID minid = String. Return results immediately newer than this ID

What should be the difference between “newer” and “immediately newer”?

Suppose the user has 10 notifications, numbered 1 through 10.

  1 2 3 4 5 6 7 8 9 10
  ^                 ^
  |                 |
oldest            newest

They make the request:

/api/v1/notifications?since_id=5&limit=2

This means they want two (limit=2) notifications that are newer than notification 5 (since_id=5).

The server returns notifications 9 and 10, as these are the two newest notifications.

Think of since_id as “Start at the newest notification, count backwards limit notifications, and return from there”.

Same setup, but the request is now:

/api/v1/notifications?min_id=5&limit=2

This means they want two (limit=2) notifications that are immediately newer than notification 5 (min_id=5).

The server returns notifications 6 and 7, as these are the two notifications immediately newer than 5.

Think of min_id as “Start at min_id, then return the next limit notifications”.

Given that min_id was added to the API after since_id was my working theory is that this was a design flaw in the API, which min_id fixed after the API was first released.

Under the hood we're updating the Tusky design to make the code easier to work with and reason about, particularly with how timelines are handled.

A timeline is any list of posts. What you see on the Home tab is a timeline, so is the Notifications tab, Local, Federated, search results, trending tags, all of it.

This is a big task, and trying to do it to all timelines at once would be very difficult. So the work is broken up to focus on one or more timelines at a time. This also means that if there's anything that turns out to be a bad idea we can fix it on the affected timeline before it affects everything.

The timeline on the Notifications tab is the first one to get these changes.

So what's changed?

Missing posts

There have been occasional reports about posts missing from timelines (https://github.com/tuskyapp/Tusky/issues/3505, https://github.com/tuskyapp/Tusky/issues/3320).

This is fixed with the re-write. As I say, initially for Notifications, but on all timelines in the next major release.

No more “Load more”

The “Load more” list entry you might see as you scroll through your notifications? That's gone. Notifications now load automatically as you scroll through the list.

You can jump to the top of the loaded list by tapping the icon in the tab header (that's not new in this release). v22.1 will add a menu item to jump to the most recent notifications, even if they haven't been loaded yet.

Remembering the reading position

Your “reading position” is remembered. If you leave the tab and come back (even if you fully exit the app) you'll be restored back to the notification you were last reading.

If that notification no longer exists (maybe you dismissed it, or changed your filters so it's no longer visible) then Tusky tries to put you as close as possible to that position.

Better error handling

Your server might not be available. Maybe it's down for maintenance, or your phone doesn't have Internet connectivity. Tusky now handles this more gracefully, with better options to recover.

If you're scrolling through notifications and the next set fail to load the current set remain, and you'll see (a) more detail about the error, and (b) an option to retry. The error details are important, as they give you a better idea of what the problem is and how you might fix it.

If you interact with a post in the notifications tab (boost, favourite, bookmark, etc) and that fails you'll also be told why, with a “Retry” option.

Previously those failures could be invisible, and lead to user feedback like “I bookmarked this post, the icon changed, but it's not in my bookmarks”. That could happen because the “bookmark this post” request was sent to the server but the server didn't receive it. Tusky optimistically updated the UI to show the bookmark had succeeded, but didn't update the UI when it failed.

This was me, in https://github.com/tuskyapp/Tusky/pull/3159.

Tusky v22 changes how Android notifications are displayed.

To understand why the behaviour is the way it is it helps to know a little bit about how Android allows apps to send notifications. Strap in, this gets a little long.

Tusky notifications are sent to Android notification “channels”. Tusky creates one channel for each type of Mastodon notification you can get (boosts, favourites, mentions, etc).

Each channel is associated with a Tusky account, so if you're logged in with two accounts you'll have two channels for boosts (one per account), two channels for mentions, etc.

You can see all of this in “Account preferences > Notifications”.

Each channel has its own controls for how notifications appear. Do they pop up on screen? Do they make a sound? Do they ignore do-not-disturb, etc? That's so you can choose to get a sound if someone mentions you, but no notification at all if someone new follows you (for example). All of that is also controlled from “Account preferences > Notifications”.

Android apps can not see what the user has set their notificat