User accounts notifications (#164)

* feat: squash all commits for notification demo

fix: add notification and template choice

feat: sketch mutation for setting the prefeences

fix: arguments for preference selection function

feat: add runtime notification data and notification preferences

fix: notification for runtime events

fix: add notification preference mutation

feat: add email strategy

fix: split Notification into onchain and offchain

fix: add channel created event notifiation

feat: distinguisher between member and channel notifiation

feat: enable offchain notification entities export

fix: notification preferences resolver

fix: migration

feat: default notification preferences

feat: start adding tests

fix: tests

fix: tests

fix: notificatino preference mutation and complete test

fix: add support for offchain dat

fix: missc

fix: format & fix build errors

fix: build and migrations

fix: bugs in setting notifications

fix: add case for notification mutation status

fix: build errors

fix: format

fix: data

* fix: remove non needed entity for migration

export / import scripts allows to specify fields of entities which will be migrated
In this case runtime notification will be migrated partially by offchain and partiall
by handlers

* feat: mark notifications as read

* fix: runtime notification migration generation

* feat: add event notification for video posted

* feat: add video comment

* feat: add chanel verification mutation

* feat: video excluded from app notifications

* feat: channel exclusion

* fix: Channel exclusion

* feat: notification deposit for video reaction

* feat: reaction for comment notification

* feat: refactor and add channel excluded mail template

* fix: email

* feat: regenerate migrations

fix: notifications

fix: misc

* fix: WS_SOURCE for archive

* feat: boilerplate code for notification testing

fix: gql subscriptions

* ♻️ Refactor into object mother like context

* 🩹 Fix: graphql client now working

* 🐛 Logged in client able to send mutation

* ✏️ Fix: set missing notification preferences

* 🩹 Match notification pref names with design

* 🩹 Match notification pref names with design

* 🩹 Fix: SetAccountNotificationPreference resolver

* ✅ Test: Channel created notification

* ✅ Test: FolloChannel notificatino

* ♻️ Ref: notification deposit function

* ✅ Test: Video posted

* ♻️ Ref: addNotification and notification type

* feat: 🎨 create extrinsic file and add make payment to channel

* feat: 🎨 add extra extrinsic for auction notifications

* feat: 🎨 add react to video, comment to video and send payment to channel extrinsics

* refactor: ✨ refactoring code in order to account for manual tests and nft issuance tx status

* feat: 🎨 improve test by adding start auction and start offer

* docs: 📝 review auction bid notification cases and added comments

* fix: 🐛 notify creator on video comment

* fix: 💚 build error

* feat: 🎨 add notification for featured video / nft

* fix: 🐛 channel follower user id

* fix: 🐛 review notification pref parsing

* feat: 🎨 migrate Account to Account with pref (all enabled)

* feat: 🎨 migration from Channel to Channel with verification

* feat: 🎨 offchain notifcations creation timestmap

* refactor: ♻️ Notification code rework to accomodate design specs

* fix: 🎨 notification delivery and fields

* feat: 🎨 no separate notifications

* migration order

* email function rework

* notifications rework

* refactor: ♻️ unify notifications

* fix: 💚 fix notification build error after schema changes

* feat: 🚚 rename and re org utils/notification directory

* fix: 💚 mappings/content/channels tsc errors

* refactor: 💚 fix content/comments and reactions / videos

* fix: 💚 fix content / nft notification deposit

* fix: 💚 channel / resolvers notification deposit

* fix: 💚 Video resolver notification deposit

* fix: 💚 admin resolvers

* fix: 💚 fix direct member payment notification deposit

* fix: 💚 fix notification resolvers

* feat: 🎨 add app root domain

* feat: 🎨 add notification links

* feat: 🎨 notification links

* feat: 🎨 set resolver for setting app_root_domain and notification center

* feat: 🎨 add email content with some formatting

* feat: 🎨 add verify channel resolver with approporiate ChannelVerification entity

* fix: 🔥 no events or notifications are removed

no events or notifications will be removed, since they are now tied to an account and at the mement we don't support account deletion

* feat: 🎨 add royalty payment

* fix: 💚 build errors

* fix: 🩹 patch squid middleware to return 401 for unauth

* fix: 💚 fix ci build

* feat: 🎨 migrate next entity id for account

* temporarely hide integration tests

* fix gitignore

* fix: 💚 eslint fixes

* prettier

* chore: 💚 generate migration data and bump versions

* feat: add channel verification to hidden entities

* feat: 🐛 migrations

* fix: 💚 CI checks on migrations

* notifications visible to accounts

* feat: 🔥 re-establish old visibility policy for notificatinos

* fix: misc

* fix: 🐛 misc fixes for migrations

* feat: 🎨 rename auctionExpired -> timedAuctionExpired and add notificatino for channel suspended

* feat: 🎨 channel suspension resolver

* feat: ✨ better emails

* fix: 🐛 royalty computation and notification id tag

* feat: 🎨 add notification timestamp

* fix: 🐛 purchased nft / channel payment / funds withdrawn notification text

* fix: 🐛 resolvers

* fix: 🐛 VideoHero and channel suspension resolvers

* feat: 💚 fix ci build

* fix: 💚 generate Data.js and add suspension to hidden entities

* fix: 🎨 add channel_suspended to migrated entities

* style: 💄 better name for computeRoyalty args

* fix: 💚 post rebase fixes

* fix: 💚 post rebase fixes

* fix: 💚 post rebase fixes

* Update src/mappings/content/nft.ts

Co-authored-by: attemka <attemka@gmail.com>

* Update src/mappings/content/nft.ts

Co-authored-by: attemka <attemka@gmail.com>

* Update schema/hidden.graphql

Co-authored-by: Zeeshan Akram <37098720+zeeshanakram3@users.noreply.github.com>

* Update schema/hidden.graphql

Co-authored-by: Zeeshan Akram <37098720+zeeshanakram3@users.noreply.github.com>

* Update schema/hidden.graphql

Co-authored-by: Zeeshan Akram <37098720+zeeshanakram3@users.noreply.github.com>

* Update schema/hidden.graphql

Co-authored-by: Zeeshan Akram <37098720+zeeshanakram3@users.noreply.github.com>

* Update src/mappings/content/channel.ts

Co-authored-by: Zeeshan Akram <37098720+zeeshanakram3@users.noreply.github.com>

* Update src/mappings/content/channel.ts

Co-authored-by: Zeeshan Akram <37098720+zeeshanakram3@users.noreply.github.com>

* Update src/server-extension/resolvers/NotificationResolver/index.ts

Co-authored-by: Zeeshan Akram <37098720+zeeshanakram3@users.noreply.github.com>

* Update package.json

Co-authored-by: Zeeshan Akram <37098720+zeeshanakram3@users.noreply.github.com>

* Update src/server-extension/resolvers/VideosResolver/index.ts

Co-authored-by: Zeeshan Akram <37098720+zeeshanakram3@users.noreply.github.com>

* Update src/server-extension/resolvers/ChannelsResolver/index.ts

Co-authored-by: Zeeshan Akram <37098720+zeeshanakram3@users.noreply.github.com>

* Update schema/hidden.graphql

Co-authored-by: Zeeshan Akram <37098720+zeeshanakram3@users.noreply.github.com>

* fix: 🐛 notification created only once when orion_db is active

* fix: 🎨 misc PR comment address

added non-visibility of notification_delivery_* entities

* feat: 🎨 exported state entity migrated

* fix: 🎨 remove joystream patched types

* fix: 🎨 add ypp status to better match the YTS code

* style: 💄 refactor code for page links

* fix: 🎨 setup text for notifications & regenerate schema

* fix: 🐛 PR comments

feat: ♻️ refactor mail code

fix: ⚡ royalty price precision

fix: 🐛 PR comments

perf: ⚡ add max number of concurrent promises for fetching channel followers

fix: 🐛 PR comments

fix: 🐛 PR comments

fix: 🐛 PR comments

regenerate data

Update src/mappings/content/channel.ts

Co-authored-by: Zeeshan Akram <37098720+zeeshanakram3@users.noreply.github.com>

Update src/server-extension/resolvers/VideosResolver/index.ts

Co-authored-by: Zeeshan Akram <37098720+zeeshanakram3@users.noreply.github.com>

fix: 🐛 overlay vs orion_db for notification

Now notification are first written in the overlay (if deposited by processor) otherwise they are written directly into orion_db

fix: 🐛 overlay vs orion_db for notification

Now notification are first written in the overlay (if deposited by processor) otherwise they are written directly into orion_db

fix: 🐛 overlay vs orion_db for notification

Now notification are first written in the overlay (if deposited by processor) otherwise they are written directly into orion_db

fix: 🐛 separate mail sent by processor and those sent by server due to overlay presence

fix: 🐛 notification & notification on chain delivery mail deposit

fix: 🐛 notification & notification on chain delivery mail deposit

fix: 🐛 notification & notification on chain delivery mail deposit

* fix: 🐛 rework channel notification data

edit notifications link and text after rework #2

review addNotifications and start fixing channel

comments and Reaction fixes

fix metadata

fix video mappings

fix utils

fix: nft mappings

fix: admin resolvers

fix: video resolvers

fix channel resolvers

* fix: 🐛 rework channel notification data

edit notifications link and text after rework #2

review addNotifications and start fixing channel

comments and Reaction fixes

fix metadata

fix video mappings

fix utils

fix: nft mappings

fix: admin resolvers

fix: video resolvers

fix channel resolvers

* fix: 🐛 rework notification data

edit notifications link and text after rework #2

review addNotifications and start fixing channel

comments and Reaction fixes

fix metadata

fix video mappings

fix utils

fix: nft mappings

fix: admin resolvers

fix: video resolvers

fix channel resolvers

* fix: 🐛 rework notification data

edit notifications link and text after rework #2

review addNotifications and start fixing channel

comments and Reaction fixes

fix metadata

fix video mappings

fix utils

fix: nft mappings

fix: admin resolvers

fix: video resolvers

fix channel resolvers

* fix: 🐛 rework notification data

edit notifications link and text after rework #2

review addNotifications and start fixing channel

comments and Reaction fixes

fix metadata

fix video mappings

fix utils

fix: nft mappings

fix: admin resolvers

fix: video resolvers

fix channel resolvers

* fix: 🐛 rework notification data

edit notifications link and text after rework #2

review addNotifications and start fixing channel

comments and Reaction fixes

fix metadata

fix video mappings

fix utils

fix: nft mappings

fix: admin resolvers

fix: video resolvers

fix channel resolvers

* fix: 🐛 rework notification data

edit notifications link and text after rework #2

review addNotifications and start fixing channel

comments and Reaction fixes

fix metadata

fix video mappings

fix utils

fix: nft mappings

fix: admin resolvers

fix: video resolvers

fix channel resolvers

---------

Co-authored-by: attemka <attemka@gmail.com>
Co-authored-by: Zeeshan Akram <37098720+zeeshanakram3@users.noreply.github.com>

.env

@@ -51,6 +51,10 @@ EMAIL_CONFIRMATION_TOKEN_RATE_LIMIT=5
 ACCOUNT_OWNERSHIP_PROOF_EXPIRY_TIME_SECONDS=300 # 5 minutes
 COOKIE_SECRET=this-is-not-so-secret-change-it
 
+# Notification related variables
+APP_ROOT_DOMAIN=gleev.xyz
+NOTIFICATION_CENTER_PATH=notification-center # https://${APP_ROOT_DOMAIN}/${NOTIFICATION_CENTER_PATH}
+
 TRUST_PROXY=uniquelocal
 
 # Sendgrid API
@@ -59,4 +63,4 @@ SENDGRID_FROM_EMAIL=gateway@example.com
 
 # Debug settings
 SQD_DEBUG=api:*
-OPENAPI_PLAYGROUND=true
+OPENAPI_PLAYGROUND=true

.github/workflows/tests.yml

@@ -2,8 +2,31 @@ name: Tests
 on: [push, pull_request]
 
 jobs:
+  migrations:
+    name: Run migrations
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Use Node.js 16.x
+        uses: actions/setup-node@v1
+        with:
+          node-version: 16.x
+      - name: Create joystream_default network
+        run: docker network create joystream_default
+      - name: Prepare workspace
+        run: make prepare
+      - name: Start db
+        run: docker-compose up -d orion_db
+      - name: Wait for db
+        run: |
+          while ! docker-compose exec -T orion_db pg_isready -U postgres -p 23798; do
+            sleep 1
+          done
+      - name: Run migrations
+        run: make migrate
   auth-api:
     name: Auth API tests
+    needs: [migrations]
     runs-on: ${{ matrix.os }}
     strategy:
       matrix:

.gitignore

@@ -5,9 +5,10 @@
 /**Versions.jsonl
 
 # IDE files
+/.vscode
 /.idea
 src/model/generated
 /schema.graphql
 /db/persisted
 /scripts/orion-v1-migration/data
-/db/export
+/db/export

.vscode/tasks.json

@@ -0,0 +1,16 @@
+{
+  "version": "2.0.0",
+  "tasks": [
+    {
+      "type": "npm",
+      "script": "lint",
+      "problemMatcher": [],
+      "label": "npm: lint",
+      "detail": "eslint --ext .ts ./src",
+      "group": {
+        "kind": "test",
+        "isDefault": true
+      }
+    }
+  ]
+}

archive/.env

@@ -1,4 +1,4 @@
 DB_PORT=12345
 GATEWAY_PORT=8888
-WS_SOURCE=wss://rpc.joystream.org:9944
-EXPLORER_PORT=4444
+WS_SOURCE=ws://joystream-node:9944
+EXPLORER_PORT=4444

assets/patches/@subsquid+graphql-server+3.3.2.patch

@@ -56,3 +56,23 @@ index 04e74c3..e95a07b 100644
          };
      }
      async createTypeormConnection(options) {
+diff --git a/node_modules/@subsquid/graphql-server/src/check.ts b/node_modules/@subsquid/graphql-server/src/check.ts
+index 51eee8d..683847b 100644
+--- a/node_modules/@subsquid/graphql-server/src/check.ts
++++ b/node_modules/@subsquid/graphql-server/src/check.ts
+@@ -48,6 +48,15 @@ export function createCheckPlugin(requestCheck: RequestCheckFunction, model: Mod
+                         model
+                     })
+                     if (typeof ok == 'string') {
++                        if (ok === 'Unauthorized') {
++                            return {
++                                errors: [{ message: 'Unauthorized'}],
++                                http: {
++                                    status: 401,
++                                    headers: new Headers(),
++                                },
++                            }
++                        }
+                         return {errors: [{message: ok}]}
+                     } else if (ok) {
+                         return null

db/migrations/2000000000000-Views.js

@@ -63,19 +63,24 @@ module.exports = class Views2000000000000 {
         `("type"->>'channel' IS NULL OR EXISTS(SELECT 1 FROM "channel" WHERE "id"="type"->>'channel'))`,
         `("type"->>'video' IS NULL OR EXISTS(SELECT 1 FROM "video" WHERE "id"="type"->>'video'))`
       ],
-      notification: [`EXISTS(SELECT 1 FROM "event" WHERE "id"="event_id")`],
       nft_history_entry: [`EXISTS(SELECT 1 FROM "event" WHERE "id"="event_id")`],
       nft_activity: [`EXISTS(SELECT 1 FROM "event" WHERE "id"="event_id")`],
       // HIDDEN entities
       video_view_event: ['FALSE'],
       channel_follow: ['FALSE'],
       report: ['FALSE'],
+      exclusion: ['FALSE'],
       session: ['FALSE'],
+      notification_email_delivery: ['FALSE'],
+      notification_in_app_delivery: ['FALSE'],
+      channel_verification: ['FALSE'],
+      channel_suspension: ['FALSE'],
       user: ['FALSE'],
       account: ['FALSE'],
       token: ['FALSE'],
       nft_featuring_request: ['FALSE'],
       gateway_config: ['FALSE'],
+      notification: ['FALSE'],
     }
   }