Skip to content
Extraits de code Groupes Projets

Comparer les révisions

Les modifications sont affichées comme si la révision source était fusionnée avec la révision cible. En savoir plus sur la comparaison des révisions.

Source

Sélectionner le projet cible
No results found
Sélectionner une révision Git
Loading items

Cible

Sélectionner le projet cible
  • arise/matrix-appservice-discord
  • Deurstann/matrix-appservice-discord
2 résultats
Sélectionner une révision Git
Loading items
Afficher les modifications
Validations sur la source (31)
Affichage de avec 256 ajouts et 99 suppressions
name: Container Image
on:
push:
branches: [ develop ]
tags: [ 'v*' ]
pull_request:
branches: [ develop ]
env:
IMAGE_NAME: ${{ github.repository }}
REGISTRY: ghcr.io
jobs:
push:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v2
- name: Log into registry ${{ env.REGISTRY }}
uses: docker/login-action@v1
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract Docker metadata
id: meta
uses: docker/metadata-action@v3
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
- name: Build and push Docker image
uses: docker/build-push-action@v3
with:
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
......@@ -17,7 +17,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
node_version: [14, 16, 18]
node_version: [16, 18]
steps:
- uses: actions/checkout@v2
- name: Use Node.js ${{ matrix.node_version }}
......
......@@ -53,3 +53,9 @@ build
*.db
*.db.backup
.vscode/
*.code-workspace
# Local History for Visual Studio Code
.history/
engine-strict = true
@mx-puppet:registry="https://gitlab.com/api/v4/packages/npm/"
3.1.1 (2022-11-10)
==================
Bugfixes
--------
- Fix a crash caused by processing metrics for Matrix events. ([\#869](https://github.com/matrix-org/matrix-appservice-discord/issues/869))
3.1.0 (2022-11-03)
==================
Features
--------
- Adds a config value, in order to disable forwarding room topic changes from Matrix to Discord (`disableRoomTopicNotifications`, false by default). ([\#836](https://github.com/matrix-org/matrix-appservice-discord/issues/836))
Bugfixes
--------
- Include the domain name in the regular expression. ([\#834](https://github.com/matrix-org/matrix-appservice-discord/issues/834))
- Remove usage of unreliable field `age` on events, allowing the bridge to work with non-Synapse homeserver implementations. ([\#842](https://github.com/matrix-org/matrix-appservice-discord/issues/842))
- Prevent crashes when handling messages sent to voice channels. ([\#858](https://github.com/matrix-org/matrix-appservice-discord/issues/858))
3.0.0 (2022-08-12)
==================
Bugfixes
--------
- Make sure we don't lose errors thrown when checking usage limits. ([\#823](https://github.com/matrix-org/matrix-appservice-discord/issues/823))
- Fix Docker instances not starting due to being unable to load a dynamic library in the latest unstable image. ([\#828](https://github.com/matrix-org/matrix-appservice-discord/issues/828))
- Remove matrix.to hyperlinks when relaying non-Discord user mentions to Discord.
Fix mentioning Matrix users in Discord. ([\#829](https://github.com/matrix-org/matrix-appservice-discord/issues/829))
Deprecations and Removals
-------------------------
- Minimum required Node.js version is now 16. ([\#825](https://github.com/matrix-org/matrix-appservice-discord/issues/825))
Internal Changes
----------------
- Remove unused variables. ([\#657](https://github.com/matrix-org/matrix-appservice-discord/issues/657))
- Add workflow for building docker images, and push new docker images to ghcr.io. ([\#826](https://github.com/matrix-org/matrix-appservice-discord/issues/826))
- Remove `git config` workaround to pull a dependency from github.com. ([\#830](https://github.com/matrix-org/matrix-appservice-discord/issues/830))
2.0.0 (2022-08-05)
==================
Improved Documentation
----------------------
- Update `CONTRIBUTING.md` guide to reference the newly-updated guide for all of the matrix.org bridge repos. ([\#794](https://github.com/matrix-org/matrix-appservice-discord/issues/794))
Deprecations and Removals
-------------------------
- Node.JS 12 is now unsupported, please upgrade to Node.JS 14 or later. Node.JS 16 becomes the new default version. ([\#811](https://github.com/matrix-org/matrix-appservice-discord/issues/811))
Internal Changes
----------------
- Add automatic changelog generation via [Towncrier](https://github.com/twisted/towncrier). ([\#787](https://github.com/matrix-org/matrix-appservice-discord/issues/787))
- Use `yarn` instead of `npm` for package management and scripts. ([\#796](https://github.com/matrix-org/matrix-appservice-discord/issues/796))
- Add new CI workflow to check for signoffs. ([\#818](https://github.com/matrix-org/matrix-appservice-discord/issues/818))
FROM node:16-alpine AS BUILD
FROM node:16-slim AS BUILD
COPY . /tmp/src
# install some dependencies needed for the build process
RUN apk add --no-cache -t build-deps make gcc g++ python ca-certificates libc-dev wget git
RUN apt update && apt install -y build-essential make gcc g++ python3 ca-certificates libc-dev wget git
RUN cd /tmp/src \
&& yarn
FROM node:16-alpine
FROM node:16-slim
ENV NODE_ENV=production
COPY --from=BUILD /tmp/src/build /build
COPY --from=BUILD /tmp/src/config /config
......
......@@ -98,7 +98,7 @@ should show up in the network list on Element and other clients.
### Setting up Discord
* Create a new application via https://discordapp.com/developers/applications
* Create a new application via https://discord.com/developers/applications
* Make sure to create a bot user. Fill in ``config.yaml``
* Run ``yarn addbot`` to get a authorisation link.
* Give this link to owners of the guilds you plan to bridge.
......
Add automatic changelog generation via [Towncrier](https://github.com/twisted/towncrier).
Update `CONTRIBUTING.md` guide to reference the newly-updated guide for all of the matrix.org bridge repos.
Use `yarn` instead of `npm` for package management and scripts.
Node.JS 12 is now unsupported, please upgrade to Node.JS 14 or later. Node.JS 16 becomes the new default version.
\ No newline at end of file
Add new CI workflow to check for signoffs.
\ No newline at end of file
......@@ -36,6 +36,8 @@ bridge:
disableJoinLeaveNotifications: false
# Disable Invite echos from matrix
disableInviteNotifications: false
# Disable Room Topic echos from matrix
disableRoomTopicNotifications: false
# Auto-determine the language of code blocks (this can be CPU-intensive)
determineCodeLanguage: false
# MXID of an admin user that will be PMd if the bridge experiences problems. Optional
......
......@@ -30,6 +30,8 @@ properties:
type: "boolean"
disableInviteNotifications:
type: "boolean"
disableRoomTopicNotifications:
type: "boolean"
userActivity:
type: "object"
required: ["minUserActiveDays", "inactiveAfterDays"]
......
{
"name": "matrix-appservice-discord",
"version": "1.0.0",
"version": "3.1.1",
"description": "A bridge between Matrix and Discord",
"main": "discordas.js",
"engines": {
"npm": "please-use-yarn",
"node": "14.x - 18.x"
"node": ">=16 <=18"
},
"scripts": {
"test": "mocha -r ts-node/register test/config.ts test/test_*.ts test/**/test_*.ts",
......@@ -40,7 +40,8 @@
},
"homepage": "https://github.com/Half-Shot/matrix-appservice-discord#readme",
"dependencies": {
"better-discord.js": "git+https://github.com/Sorunome/better-discord.js.git#b5a28499899fe2d9e6aa1aa3b3c5d693ae672117",
"@deurstann/matrix-discord-parser": "1.10.7",
"better-discord.js": "github:matrix-org/better-discord.js#5024781db755259e88abe915630b7d5b3ba5f48f",
"better-sqlite3": "^7.1.0",
"command-line-args": "^5.1.1",
"command-line-usage": "^6.1.0",
......@@ -48,10 +49,8 @@
"escape-string-regexp": "^4.0.0",
"js-yaml": "^3.14.0",
"marked": "^1.2.2",
"matrix-appservice-bridge": "^3.1.0",
"matrix-discord-parser": "0.1.5",
"matrix-appservice-bridge": "^5.0.0",
"mime": "^2.4.6",
"node-html-parser": "^1.2.19",
"p-queue": "^6.4.0",
"pg-promise": "^10.5.6",
"prom-client": "^12.0.0",
......
......@@ -20,7 +20,7 @@ import { DiscordStore } from "./store";
import { DbEmoji } from "./db/dbdataemoji";
import { DbEvent } from "./db/dbdataevent";
import { DiscordMessageProcessor } from "./discordmessageprocessor";
import { IDiscordMessageParserResult } from "matrix-discord-parser";
import { IDiscordMessageParserResult } from "@deurstann/matrix-discord-parser";
import { MatrixEventProcessor, MatrixEventProcessorOpts, IMatrixEventProcessorResult } from "./matrixeventprocessor";
import { PresenceHandler } from "./presencehandler";
import { Provisioner } from "./provisioner";
......@@ -94,7 +94,6 @@ class DiscordBridgeBlocker extends BridgeBlocker {
export class DiscordBot {
private clientFactory: DiscordClientFactory;
private _bot: Discord.Client|undefined;
private presenceInterval: number;
private sentMessages: string[];
private lastEventIds: { [channelId: string]: string };
private discordMsgProcessor: DiscordMessageProcessor;
......@@ -218,7 +217,9 @@ export class DiscordBot {
if (this.config.bridge.userLimit !== null) {
log.info(`Bridge blocker is enabled with a user limit of ${this.config.bridge.userLimit}`);
this.bridgeBlocker = new DiscordBridgeBlocker(this.config.bridge.userLimit, this);
this.bridgeBlocker?.checkLimits(activeUsers);
this.bridgeBlocker?.checkLimits(activeUsers).catch(err => {
log.error(`Failed to check bridge limits: ${err}`);
});
}
}
......@@ -342,7 +343,7 @@ export class DiscordBot {
client.on("userUpdate", async (_, user) => {
try {
if (!(user instanceof Discord.User)) {
log.warn(`Ignoring update for ${user.username}. User was partial.`);
log.warn(`Ignoring update for ${(<any>user).username}. User was partial.`);
return;
}
await this.userSync.OnUpdateUser(user);
......@@ -351,7 +352,7 @@ export class DiscordBot {
client.on("guildMemberAdd", async (member) => {
try {
if (!(member instanceof Discord.GuildMember)) {
log.warn(`Ignoring update for ${member.guild.id} ${member.id}. User was partial.`);
log.warn(`Ignoring update for ${(<any>member).guild?.id} ${(<any>member).id}. User was partial.`);
return;
}
await this.userSync.OnAddGuildMember(member);
......@@ -369,7 +370,7 @@ export class DiscordBot {
client.on("guildMemberUpdate", async (_, member) => {
try {
if (!(member instanceof Discord.GuildMember)) {
log.warn(`Ignoring update for ${member.guild.id} ${member.id}. User was partial.`);
log.warn(`Ignoring update for ${(<any>member).guild.id} ${(<any>member).id}. User was partial.`);
return;
}
await this.userSync.OnUpdateGuildMember(member);
......@@ -566,7 +567,7 @@ export class DiscordBot {
return;
}
const link = `https://discord.com/channels/${chan.guild.id}/${chan.id}/${editEventId}`;
embedSet.messageEmbed.description = `[Edit](${link}): ${embedSet.messageEmbed.description}`;
embedSet.messageEmbed.description = `[Edit](<${link}>): ${embedSet.messageEmbed.description}`;
await this.send(embedSet, opts, roomLookup, event);
} catch (err) {
// throw wrapError(err, Unstable.ForeignNetworkError, "Couldn't edit message");
......@@ -1040,65 +1041,14 @@ export class DiscordBot {
}
try {
const intent = this.GetIntentFromDiscordMember(msg.author, msg.webhookID);
// Check Attachements
if (!editEventId) {
// on discord you can't edit in images, you can only edit text
// so it is safe to only check image upload stuff if we don't have
// an edit
await Util.AsyncForEach(msg.attachments.array(), async (attachment) => {
const content = await Util.DownloadFile(attachment.url);
const fileMime = content.mimeType || mime.getType(attachment.name || "")
|| "application/octet-stream";
const mxcUrl = await intent.underlyingClient.uploadContent(
content.buffer,
fileMime,
attachment.name || "",
);
const type = fileMime.split("/")[0];
let msgtype = {
audio: "m.audio",
image: "m.image",
video: "m.video",
}[type];
if (!msgtype) {
msgtype = "m.file";
}
const info = {
mimetype: fileMime,
size: attachment.size,
} as IMatrixMediaInfo;
if (msgtype === "m.image" || msgtype === "m.video") {
info.w = attachment.width!;
info.h = attachment.height!;
}
await Util.AsyncForEach(rooms, async (room) => {
const eventId = await intent.sendEvent(room, {
body: attachment.name || "file",
external_url: attachment.url,
info,
msgtype,
url: mxcUrl,
});
this.lastEventIds[room] = eventId;
const evt = new DbEvent();
evt.MatrixId = `${eventId};${room}`;
evt.DiscordId = msg.id;
evt.ChannelId = msg.channel.id;
if (msg.guild) {
evt.GuildId = msg.guild.id;
}
await this.store.Insert(evt);
this.userActivity.updateUserActivity(intent.userId);
});
});
}
if (!msg.content && msg.embeds.length === 0) {
if (!msg.content && msg.embeds.length === 0 && msg.attachments.array().length === 0) {
return;
}
const result = await this.discordMsgProcessor.FormatMessage(msg);
if (!result.body) {
if (!result.body && msg.attachments.array().length === 0) {
return;
}
if(result.body){
await Util.AsyncForEach(rooms, async (room) => {
const sendContent: IMatrixMessage = {
body: result.body,
......@@ -1106,6 +1056,30 @@ export class DiscordBot {
formatted_body: result.formattedBody,
msgtype: result.msgtype,
};
if (msg.reference) {
const storeEvent = await this.store.Get(DbEvent, { discord_id: msg.reference?.messageID });
if (storeEvent && storeEvent.Result) {
let replyToEventId: string | undefined = undefined;
while (storeEvent.Next()) {
const [eventId] = storeEvent.MatrixId.split(";");
// Try to get the "deepest" event ID if this event replaces another ID
// We need to do this since a m.in_reply_to relation requires the original event ID and not the replacement one
const { chunk } = await intent.underlyingClient.unstableApis.getRelationsForEvent(room, eventId, "m.replace");
if (!!chunk?.length) {
replyToEventId = chunk[0].content['m.relates_to'].event_id;
} else {
replyToEventId ??= eventId;
}
}
if (replyToEventId) {
sendContent["m.relates_to"] = {
"m.in_reply_to": {
event_id: replyToEventId
}
};
}
}
}
if (editEventId) {
sendContent.body = `* ${result.body}`;
sendContent.formatted_body = `* ${result.formattedBody}`;
......@@ -1151,6 +1125,59 @@ export class DiscordBot {
await afterSend(res);
}
});
}
// Check Attachements
if (!editEventId) {
// on discord you can't edit in images, you can only edit text
// so it is safe to only check image upload stuff if we don't have
// an edit
await Util.AsyncForEach(msg.attachments.array(), async (attachment) => {
const content = await Util.DownloadFile(attachment.url);
const fileMime = content.mimeType || mime.getType(attachment.name || "")
|| "application/octet-stream";
const mxcUrl = await intent.underlyingClient.uploadContent(
content.buffer,
fileMime,
attachment.name || "",
);
const type = fileMime.split("/")[0];
let msgtype = {
audio: "m.audio",
image: "m.image",
video: "m.video",
}[type];
if (!msgtype) {
msgtype = "m.file";
}
const info = {
mimetype: fileMime,
size: attachment.size,
} as IMatrixMediaInfo;
if (msgtype === "m.image" || msgtype === "m.video") {
info.w = attachment.width!;
info.h = attachment.height!;
}
await Util.AsyncForEach(rooms, async (room) => {
const eventId = await intent.sendEvent(room, {
body: attachment.name || "file",
external_url: attachment.url,
info,
msgtype,
url: mxcUrl,
});
this.lastEventIds[room] = eventId;
const evt = new DbEvent();
evt.MatrixId = `${eventId};${room}`;
evt.DiscordId = msg.id;
evt.ChannelId = msg.channel.id;
if (msg.guild) {
evt.GuildId = msg.guild.id;
}
await this.store.Insert(evt);
this.userActivity.updateUserActivity(intent.userId);
});
});
}
MetricPeg.get.requestOutcome(msg.id, true, "success");
} catch (err) {
MetricPeg.get.requestOutcome(msg.id, true, "fail");
......@@ -1168,6 +1195,9 @@ export class DiscordBot {
if (storeEvent && storeEvent.Result) {
while (storeEvent.Next()) {
const matrixIds = storeEvent.MatrixId.split(";");
if(!oldMsg.content && oldMsg.attachments.array().length>0){
newMsg.content = newMsg.content + " " + oldMsg.attachments.array()[0].url;
}
await this.OnMessage(newMsg, matrixIds[0]);
return;
}
......@@ -1235,7 +1265,9 @@ export class DiscordBot {
await this.store.storeUserActivity(userId, state.dataSet.users[userId]);
}
log.verbose(`Checking bridge limits (${state.activeUsers} active users)`);
this.bridgeBlocker?.checkLimits(state.activeUsers);
this.bridgeBlocker?.checkLimits(state.activeUsers).catch(err => {
log.error(`Failed to check bridge limits: ${err}`);
});;
MetricPeg.get.setRemoteMonthlyActiveUsers(state.activeUsers);
}
}
......@@ -1278,7 +1310,9 @@ class AdminNotifier {
await this.client.inviteUser(mxid, roomId);
} catch (err) {
log.verbose(`Failed to invite ${mxid} to ${roomId}, cleaning up`);
this.client.leaveRoom(roomId); // no point awaiting it, nothing we can do if we fail
this.client.leaveRoom(roomId).catch(err => {
log.error(`Failed to clean up to-be-DM room ${roomId}: ${err}`);
});
throw err;
}
......
......@@ -80,7 +80,7 @@ export class DiscordBridgeConfig {
}
}
class DiscordBridgeConfigBridge {
export class DiscordBridgeConfigBridge {
public domain: string;
public homeserverUrl: string;
public port: number;
......@@ -97,6 +97,7 @@ class DiscordBridgeConfigBridge {
public disableHereMention: boolean = false;
public disableJoinLeaveNotifications: boolean = false;
public disableInviteNotifications: boolean = false;
public disableRoomTopicNotifications: boolean = false;
public determineCodeLanguage: boolean = false;
public activityTracker: UserActivityTrackerConfig = UserActivityTrackerConfig.DEFAULT;
public userLimit: number|null = null;
......
......@@ -25,7 +25,7 @@ const log = new Log("DbRoomStore");
/**
* A RoomStore compatible with
* https://github.com/matrix-org/matrix-appservice-bridge/blob/master/lib/components/room-bridge-store.js
* https://github.com/matrix-org/matrix-appservice-bridge/blob/master/src/components/room-bridge-store.ts
* that accesses the database instead.
*/
......@@ -122,8 +122,6 @@ export class DbRoomStore {
}
public async upsertEntry(entry: IRoomStoreEntry) {
const promises: Promise<void>[] = [];
const row = (await this.db.Get("SELECT * FROM room_entries WHERE id = $id", {id: entry.id})) || {};
if (!row.id) {
......
......@@ -16,7 +16,6 @@ limitations under the License.
import {IDbSchema} from "./dbschema";
import {DiscordStore} from "../../store";
import { Log } from "../../log";
export class Schema implements IDbSchema {
public description = "create room store tables";
......
......@@ -39,7 +39,7 @@ const commandOptions = [
{ name: "help", alias: "h", type: Boolean },
];
function generateRegistration(opts, registrationPath: string): void {
function generateRegistration(opts, registrationPath: string, config: DiscordBridgeConfig): void {
if (!opts.url) {
throw Error("'url' not given in command line opts, cannot generate registration file");
}
......@@ -52,14 +52,14 @@ function generateRegistration(opts, registrationPath: string): void {
aliases: [
{
exclusive: true,
regex: "#_discord_.*",
regex: '#_discord_.+:' + config.bridge.domain,
},
],
rooms: [ ],
users: [
{
exclusive: true,
regex: "@_discord_.*",
regex: '@_discord_.+:' + config.bridge.domain,
},
],
},
......@@ -125,13 +125,6 @@ async function run(): Promise<void> {
const configPath = opts.config || "config.yaml";
const registrationPath = opts.file || "discord-registration.yaml";
if (opts["generate-registration"]) {
if (fs.existsSync(registrationPath)) {
throw Error("Not writing new registration file, file already exists");
}
generateRegistration(opts, registrationPath);
return;
}
const config = new DiscordBridgeConfig();
const readConfig = yaml.safeLoad(fs.readFileSync(configPath, "utf8"));
......@@ -140,6 +133,15 @@ async function run(): Promise<void> {
}
config.applyConfig(readConfig);
config.applyEnvironmentOverrides(process.env);
if (opts["generate-registration"]) {
if (fs.existsSync(registrationPath)) {
throw Error("Not writing new registration file, file already exists");
}
generateRegistration(opts, registrationPath, config);
return;
}
Log.Configure(config.logging);
const port = opts.port || config.bridge.port;
if (!port) {
......