NSFW in For You

NSFW content has been a source of complaints and "show less like this" interactions on 💖 For You .

Let's look at the data. I started logging the labels applied to the posts shown in For You and here are some stats for content with no labels vs with labels from this morning:

flag impressions like_% show_less_% show_more_%
-------------------- ------------ -------- ------------ ------------
clean 11702220 1.30 0.0144 0.0031

porn 1295504 2.01 0.0313 0.0124
sexual 509315 1.64 0.0210 0.0100
nudity 110215 1.73 0.0354 0.0145
sexual_figurative 29592 1.69 0.0169 0.0169
graphic_media 19093 1.60 0.0576 0.0105

Indeed, the content with the labels is ~2x more likely to get "show less like this" (0.03% vs 0.014%). Interestingly, "graphic media" gets the highest rate of "show less". However, the sample size for this label is too small to draw strong conclusion.

What is more interesting is that labeled content has a significantly higher like rate (1.3% -> 2%) and a 4x higher rate of "show more like this" (0.0031% -> ~0.012%). The "show more" increase could be due to the private nature of this interaction - nobody else but the feed operator gets to know about this and users could be more comfortable using this vs publicly liking.

I long wanted to find out how to make the proper use of the content labels. When I created https://foryou.club/also-liked I saw that when you enter a sfw post as the seed item you would often see nsfw content in the recommendations just because people who happened to like it also liked nsfw posts.

In such a simple "also liked" case I decided to filter out content that is more nsfw than the seed item. If the user has entered a sfw seed post then they have not positively expressed that they want nsfw content. Before we show nsfw content we probably want some explicit signal from the user that they are seeking such content.

Would such logic work in For You more generally? Often people complain that they get nsfw content when they haven't liked such content at all. The simplest thing we could do is find the labels of all content the user has liked in the past and not show content that is "more nsfw" than their past likes. For example, if the user liked at most "sexual figurative" then we would keep recommending such content but we would not expose them to "porn".

The labels though don't capture the full spectrum of nsfw flavors. A user may like one type of "sexual" content but not all. Could we capture this?

Here is an idea - show a nsfw post only if it was liked by at least one co-liker with whom you share a like on a post with the same nsfw label.

Suppose we are generating For You for Alice:

If Alice liked sfw posts that Bob liked then when Bob likes nsfw posts we ignore them because Alice has not shown that she shares Bob's nsfw tastes.

If Alice likes nsfw posts that Carol liked then other nsfw posts that Carol liked get through.

I added this logic to For You and you can test it out in the playground by enabling this option:

old logic: https://foryou.club/playground

new logic: https://foryou.club/playground?filter_nsfw=1

A/B test

As always, to see if this change is any good I have started an A/B test where 50% of the users will see the old logic and 50% will have the new logic. I expect that in the "new logic" group For You will show less nsfw content to people who don't want it and it would be reflected in the metrics:

we would see a lower rate of "show less" for nsfw content

the volume of nsfw labeled post impressions is not affected too much

overall like rates per feed load and per user stay the same

the number of feed loads per user is slightly higher because users are not turned away when then come across unwanted content