Recent changes to this wiki:
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn index b2c4144..46984fc 100644 --- a/TheCaseForFromRewriting.mdwn +++ b/TheCaseForFromRewriting.mdwn @@ -60,7 +60,7 @@ Receiver side tools including spamassassin collect validation signals from the a [Yahoo, google, apple, and microsoft](https://dmarcian.com/yahoo-and-google-dmarc-required/) are increasingly demanding DMARC alignment from incoming mail. -Sourceware's postfix mail server includes "milter" plugins for all of the above mail server tech. Sourceware publishes DKIM/DMARC/ARC policies and keys in DNS. For the moment, its DMARC policy is [p=none, but it should change to p=quarantine or p=reject](https://mxtoolbox.com/dmarc/details/dmarc-tags/dmarc-policy-options) to improve deliverability and further forwarding of our email. +Sourceware's postfix mail server includes "milter" plugins for all of the above mail server tech. Sourceware publishes DKIM/DMARC/ARC policies and keys in DNS. Our DMARC policy is [transitioning](https://sourceware.org/bugzilla/show_bug.cgi?id=33719) to improve deliverability and further forwarding of our email. ## Incoming mail
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn index e658cf0..b2c4144 100644 --- a/TheCaseForFromRewriting.mdwn +++ b/TheCaseForFromRewriting.mdwn @@ -60,7 +60,7 @@ Receiver side tools including spamassassin collect validation signals from the a [Yahoo, google, apple, and microsoft](https://dmarcian.com/yahoo-and-google-dmarc-required/) are increasingly demanding DMARC alignment from incoming mail. -Sourceware's postfix mail server includes "milter" plugins for all of the above mail server tech. +Sourceware's postfix mail server includes "milter" plugins for all of the above mail server tech. Sourceware publishes DKIM/DMARC/ARC policies and keys in DNS. For the moment, its DMARC policy is [p=none, but it should change to p=quarantine or p=reject](https://mxtoolbox.com/dmarc/details/dmarc-tags/dmarc-policy-options) to improve deliverability and further forwarding of our email. ## Incoming mail
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn index 037eb39..e658cf0 100644 --- a/TheCaseForFromRewriting.mdwn +++ b/TheCaseForFromRewriting.mdwn @@ -102,6 +102,10 @@ The effect is that the user-visible "From:" address identifies the name of the p Yes, it is. +## And it's fundamentally wrong and evil. + +It is a technical compromise with trade-offs, including breaking with some traditions. + ## And it breaks git patches. Yes, if you just "git am" received emails, the Author field of the git commit may be screwed up.
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn index cb75683..037eb39 100644 --- a/TheCaseForFromRewriting.mdwn +++ b/TheCaseForFromRewriting.mdwn @@ -50,7 +50,7 @@ Additional cryptography layers added by the end-users via PGP or S/MIME but thes ### IP reputation -Mail originators are scored by multiple agencies about the quantity of traffic and spam they pass. This reputation is available via DNS queries to recipients as an additional signal. Sourceware has signed up with [https://www.dnswl.org/], a commonly used whitelist. +Mail originators are scored by multiple agencies about the quantity of traffic and spam they pass. This reputation is available via DNS queries to recipients as an additional signal. Sourceware has signed up with [DNSWL](https://www.dnswl.org/), a commonly used whitelist. ### spam filtering
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn
index e3199e0..cb75683 100644
--- a/TheCaseForFromRewriting.mdwn
+++ b/TheCaseForFromRewriting.mdwn
@@ -111,7 +111,8 @@ However, you can
* hand-amend the author field of the commit ("git commit --amend --author=FOO")
* instruct your contributors to send git patches with a safe From: author field in the patch body by adding this into their .gitconfig:
-.
+:
+
[format]
from = true
forceInBodyFrom = true
This reverts commit 0f2c54641082bd9288012e12a3eb3889a03c4e67
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn index ac6eb9c..e3199e0 100644 --- a/TheCaseForFromRewriting.mdwn +++ b/TheCaseForFromRewriting.mdwn @@ -60,7 +60,7 @@ Receiver side tools including spamassassin collect validation signals from the a [Yahoo, google, apple, and microsoft](https://dmarcian.com/yahoo-and-google-dmarc-required/) are increasingly demanding DMARC alignment from incoming mail. -Sourceware's postfix mail server includes "milter" plugins for all of the above mail server tech: enforcing compliance for incoming email, and assuring compliance on *some* outgoing mail. +Sourceware's postfix mail server includes "milter" plugins for all of the above mail server tech. ## Incoming mail
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn
index b9bc67a..ac6eb9c 100644
--- a/TheCaseForFromRewriting.mdwn
+++ b/TheCaseForFromRewriting.mdwn
@@ -111,8 +111,7 @@ However, you can
* hand-amend the author field of the commit ("git commit --amend --author=FOO")
* instruct your contributors to send git patches with a safe From: author field in the patch body by adding this into their .gitconfig:
-
- \# add From: author into patch body to work around From:-rewriting mailing lists
+.
[format]
from = true
forceInBodyFrom = true
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn
index 5ab9a8a..b9bc67a 100644
--- a/TheCaseForFromRewriting.mdwn
+++ b/TheCaseForFromRewriting.mdwn
@@ -112,7 +112,7 @@ However, you can
* instruct your contributors to send git patches with a safe From: author field in the patch body by adding this into their .gitconfig:
- # add From: author into patch body to work around From:-rewriting mailing lists
+ \# add From: author into patch body to work around From:-rewriting mailing lists
[format]
from = true
forceInBodyFrom = true
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn
index 5d3ea82..5ab9a8a 100644
--- a/TheCaseForFromRewriting.mdwn
+++ b/TheCaseForFromRewriting.mdwn
@@ -112,8 +112,7 @@ However, you can
* instruct your contributors to send git patches with a safe From: author field in the patch body by adding this into their .gitconfig:
-\#
-
+ # add From: author into patch body to work around From:-rewriting mailing lists
[format]
from = true
forceInBodyFrom = true
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn
index a644939..5d3ea82 100644
--- a/TheCaseForFromRewriting.mdwn
+++ b/TheCaseForFromRewriting.mdwn
@@ -112,8 +112,8 @@ However, you can
* instruct your contributors to send git patches with a safe From: author field in the patch body by adding this into their .gitconfig:
-:
-
+\#
+
[format]
from = true
forceInBodyFrom = true
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn index d4d8169..a644939 100644 --- a/TheCaseForFromRewriting.mdwn +++ b/TheCaseForFromRewriting.mdwn @@ -60,7 +60,7 @@ Receiver side tools including spamassassin collect validation signals from the a [Yahoo, google, apple, and microsoft](https://dmarcian.com/yahoo-and-google-dmarc-required/) are increasingly demanding DMARC alignment from incoming mail. -Sourceware's postfix mail server includes "milter" plugins for all of the above mail server tech. +Sourceware's postfix mail server includes "milter" plugins for all of the above mail server tech: enforcing compliance for incoming email, and assuring compliance on *some* outgoing mail. ## Incoming mail
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn index 0e27c0e..d4d8169 100644 --- a/TheCaseForFromRewriting.mdwn +++ b/TheCaseForFromRewriting.mdwn @@ -58,7 +58,7 @@ Receiver side tools including spamassassin collect validation signals from the a # Policies of major mail servers -Yahoo, google, and microsoft are increasingly demanding DMARC alignment from incoming mail. +[Yahoo, google, apple, and microsoft](https://dmarcian.com/yahoo-and-google-dmarc-required/) are increasingly demanding DMARC alignment from incoming mail. Sourceware's postfix mail server includes "milter" plugins for all of the above mail server tech.
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn index e2b1a7a..0e27c0e 100644 --- a/TheCaseForFromRewriting.mdwn +++ b/TheCaseForFromRewriting.mdwn @@ -82,7 +82,13 @@ There is an inherent conflict between forwarding email through mailing lists and # The controversy: mailing list From: rewriting -The DMARC policies aim to prove to email recipients that the email either arrived directly from its From:-visible author (SPF), or may have been relayed but was unmolested on the way (DKIM). The first criterion clearly cannot be met by a mailing list relay, which leaves DKIM as the only way to assure receiving systems. However, a DKIM signature associated with a given sender needs to come from a server associated with that author (with access to the domain's private keys), which a third party mailing list host cannot. Even if the author's originating mail server does sign the outgoing message with DKIM, and sourceware relays that signature, mailman could accidentally invalidate that signature. Even quite innocent changes can mess it up, like changing message content encoding or tweaking white space. Another example is by adding list-related headers that were not there before and were ["oversigned"](https://forum.dmarcian.com/t/oversigning-headers-what-is-it-and-how-does-one-do-it/2462). Some MTA software such as Exim defaults to oversigning several list-related fields. This results in invalid DKIM signatures, and thus a DMARC failure, and thus mail bouncing. +The DMARC policies aim to prove to email recipients that the email either arrived directly from its From:-visible author (SPF), or may have been relayed but was unmolested on the way (DKIM). The first criterion clearly cannot be met by a mailing list relay, which leaves DKIM as the only way to assure receiving systems. However, a DKIM signature associated with a given sender needs to come from a server associated with that author (with access to the domain's private keys), which a third party mailing list host cannot. + +Even if the author's originating mail server does sign the outgoing message with DKIM, and sourceware relays that signature, mailman could accidentally invalidate that signature. Even quite innocent changes can mess it up, like changing message content encoding or tweaking white space. Another example is by adding list-related headers that were not there before and were ["oversigned"](https://forum.dmarcian.com/t/oversigning-headers-what-is-it-and-how-does-one-do-it/2462). Some MTA software such as Exim defaults to oversigning several list-related fields. This results in invalid DKIM signatures, and thus a DMARC failure, and thus mail bouncing. + +Consider the possibility that the originating mail server does not sign with DKIM at all. In that case, sourceware's relay breaks SPF From: alignment, and cannot sign with DKIM on behalf of the originating server (nobody can). Instant DMARC failure, mail bouncing. + +In neither of these cases did the originator system do anything wrong. In both cases, the original messages were perfectly compliant. It is sourceware's relaying operation that broke what would otherwise have passed DMARC. The policy cure to this problem is to rewrite the From: field of the email as it passes through the mailing list. This looks like:
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn index 7622bca..e2b1a7a 100644 --- a/TheCaseForFromRewriting.mdwn +++ b/TheCaseForFromRewriting.mdwn @@ -1,4 +1,4 @@ -Email used to be easy: receive it, deliver it. Then it became harder: receive it, spamassassin it, bounce or deliver it. But now, with the firehose of spam targeting billions of mail accounts, mail services have had to adapt increasingly draconian measures to filter out spam and forgery but reliably deliver real mail. +Email used to be easy: receive it, deliver it. Then it became harder: receive it, spamassassin it, bounce or deliver it. But now, with the firehose of spam targeting billions of mail accounts, mail services have had to adapt increasingly draconian measures to filter out spam and forgery but reliably deliver real email. [[!toc]]
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn
index adaee6d..7622bca 100644
--- a/TheCaseForFromRewriting.mdwn
+++ b/TheCaseForFromRewriting.mdwn
@@ -116,7 +116,7 @@ However, you can
Yes, it does.
- dmarc_moderation_action = munge_from
+ dmarc_moderation_action = Munge From
dmarc_quarantine_moderation_action = yes
dmarc_none_moderation_action = yes
- from_is_list = munge_from
+ from_is_list = Munge From
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn index 84319c5..adaee6d 100644 --- a/TheCaseForFromRewriting.mdwn +++ b/TheCaseForFromRewriting.mdwn @@ -58,7 +58,7 @@ Receiver side tools including spamassassin collect validation signals from the a # Policies of major mail servers -Yahoo, google, and microsoft are increasingly demanding DMARC alignment +Yahoo, google, and microsoft are increasingly demanding DMARC alignment from incoming mail. Sourceware's postfix mail server includes "milter" plugins for all of the above mail server tech. @@ -115,3 +115,8 @@ However, you can ## But it works. Yes, it does. + + dmarc_moderation_action = munge_from + dmarc_quarantine_moderation_action = yes + dmarc_none_moderation_action = yes + from_is_list = munge_from
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn
index 9e0c230..84319c5 100644
--- a/TheCaseForFromRewriting.mdwn
+++ b/TheCaseForFromRewriting.mdwn
@@ -111,3 +111,7 @@ However, you can
[format]
from = true
forceInBodyFrom = true
+
+## But it works.
+
+Yes, it does.
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn index 602c827..9e0c230 100644 --- a/TheCaseForFromRewriting.mdwn +++ b/TheCaseForFromRewriting.mdwn @@ -82,7 +82,7 @@ There is an inherent conflict between forwarding email through mailing lists and # The controversy: mailing list From: rewriting -The DMARC policies aim to prove to email recipients that the email either arrived directly from its From:-visible author (SPF), or may have been relayed but was unmolested on the way (DKIM). The first criterion clearly cannot be met by a mailing list relay, which leaves DKIM as the only way to assure receiving systems. However, a DKIM signature associated with a given sender needs to come from a server associated with that author (with access to the domain's private keys), which a third party mailing list host cannot. Even if the author's originating mail server does sign the outgoing message with DKIM, and sourceware relays that signature, mailman could accidentally invalidate that signature by adding list-related headers that were not there before and were ["oversigned"](https://forum.dmarcian.com/t/oversigning-headers-what-is-it-and-how-does-one-do-it/2462). Some MTA software such as Exim defaults to oversigning several list-related fields. This results in invalid DKIM signatures, and thus a DMARC failure, and thus mail bouncing. +The DMARC policies aim to prove to email recipients that the email either arrived directly from its From:-visible author (SPF), or may have been relayed but was unmolested on the way (DKIM). The first criterion clearly cannot be met by a mailing list relay, which leaves DKIM as the only way to assure receiving systems. However, a DKIM signature associated with a given sender needs to come from a server associated with that author (with access to the domain's private keys), which a third party mailing list host cannot. Even if the author's originating mail server does sign the outgoing message with DKIM, and sourceware relays that signature, mailman could accidentally invalidate that signature. Even quite innocent changes can mess it up, like changing message content encoding or tweaking white space. Another example is by adding list-related headers that were not there before and were ["oversigned"](https://forum.dmarcian.com/t/oversigning-headers-what-is-it-and-how-does-one-do-it/2462). Some MTA software such as Exim defaults to oversigning several list-related fields. This results in invalid DKIM signatures, and thus a DMARC failure, and thus mail bouncing. The policy cure to this problem is to rewrite the From: field of the email as it passes through the mailing list. This looks like:
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn index d75b6aa..602c827 100644 --- a/TheCaseForFromRewriting.mdwn +++ b/TheCaseForFromRewriting.mdwn @@ -68,11 +68,11 @@ Incoming mail is checked against all the criteria above, and positive and negati ## Outgoing mail, locally originated -Outgoing mail originating from sourceware services such as bugzilla is fully signed, protected, sealed, and is bound to be delivered. Our IP reputation appears to be fine. +Outgoing mail originating from sourceware services such as bugzilla is fully signed, protected, sealed, and is bound to be delivered. ## Outgoing mail, personally forwarded -Outgoing mail originating from forwarding by ordinary users, are subjected to SRS envelope rewriting. +Outgoing mail originating from forwarding by ordinary users, are subjected to SRS envelope rewriting, which are generally reliably received. ## Mailing list traffic, forwarded through postfix
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn
index c60d66f..d75b6aa 100644
--- a/TheCaseForFromRewriting.mdwn
+++ b/TheCaseForFromRewriting.mdwn
@@ -105,10 +105,9 @@ However, you can
* hand-amend the author field of the commit ("git commit --amend --author=FOO")
* instruct your contributors to send git patches with a safe From: author field in the patch body by adding this into their .gitconfig:
- foo
+:
+
[format]
from = true
forceInBodyFrom = true
-
- bar
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn
index fd3bede..c60d66f 100644
--- a/TheCaseForFromRewriting.mdwn
+++ b/TheCaseForFromRewriting.mdwn
@@ -105,6 +105,10 @@ However, you can
* hand-amend the author field of the commit ("git commit --amend --author=FOO")
* instruct your contributors to send git patches with a safe From: author field in the patch body by adding this into their .gitconfig:
+ foo
+
[format]
from = true
forceInBodyFrom = true
+
+ bar
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn
index c6640d9..fd3bede 100644
--- a/TheCaseForFromRewriting.mdwn
+++ b/TheCaseForFromRewriting.mdwn
@@ -105,8 +105,6 @@ However, you can
* hand-amend the author field of the commit ("git commit --amend --author=FOO")
* instruct your contributors to send git patches with a safe From: author field in the patch body by adding this into their .gitconfig:
-
[format]
from = true
forceInBodyFrom = true
-
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn
index 3b6ff62..c6640d9 100644
--- a/TheCaseForFromRewriting.mdwn
+++ b/TheCaseForFromRewriting.mdwn
@@ -104,7 +104,8 @@ However, you can
* hand-amend the author field of the commit ("git commit --amend --author=FOO")
* instruct your contributors to send git patches with a safe From: author field in the patch body by adding this into their .gitconfig:
-
+
+
[format]
from = true
forceInBodyFrom = true
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn
index c053a28..3b6ff62 100644
--- a/TheCaseForFromRewriting.mdwn
+++ b/TheCaseForFromRewriting.mdwn
@@ -4,7 +4,7 @@ Email used to be easy: receive it, deliver it. Then it became harder: receive i
# Technology background: email authentication technologies
-## [SPF](https://en.wikipedia.org/wiki/Sender_Policy_Framework)
+### [SPF](https://en.wikipedia.org/wiki/Sender_Policy_Framework)
Purpose: to verify that email for a domain is directly originating from an IP address approved by that domain.
@@ -12,7 +12,7 @@ Sender: lists names of outgoing mail servers in text records in DNS.
Receiver: verifies envelope "From " header during SMTP connection, to check IP address vs. SPF records.
-## [DKIM](https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail)
+### [DKIM](https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail)
Purpose: to verify that portions of a message/body have not been altered by intermediaries.
@@ -20,7 +20,7 @@ Sender: posts DKIM public keys for domain in DNS. Cryptographically signs a sub
Receiver: verifies each DKIM signature against received subsequent headers+body
-## [ARC](https://en.wikipedia.org/wiki/Authenticated_Received_Chain)
+### [ARC](https://en.wikipedia.org/wiki/Authenticated_Received_Chain)
Purpose: to verify that intermediate servers passed along authentication results.
@@ -28,7 +28,7 @@ Sender: adds ARC headers for authenticated emails traveling through system.
Receiver: checks ARC signatures.
-## [SRS](https://en.wikipedia.org/wiki/Sender_Rewriting_Scheme)
+### [SRS](https://en.wikipedia.org/wiki/Sender_Rewriting_Scheme)
Purpose: allow forwarded emails to trigger bounces that go back to originator
@@ -36,7 +36,7 @@ Sender: rewrites the message envelope with a temporarily valid address that's as
Receiver: replies to temporary address if an email is rejected/bounced
-## [DMARC](https://en.wikipedia.org/wiki/DMARC)
+### [DMARC](https://en.wikipedia.org/wiki/DMARC)
Purpose: to verify alignment of *user-visible* email headers & content with SPF and/or DKIM authentication.
@@ -44,15 +44,15 @@ Sender: publishes a policy record in DNS
Receiver: verifies that message was signed with DKIM and/or came directly from an approved sender for the user-visible ("From:") domain.
-## personal email cryptography
+### personal email cryptography
Additional cryptography layers added by the end-users via PGP or S/MIME but these do not really help signal their authenticity. Personal signatures are hard to verify (key distribution problem). Encryption frustrates content based analysis.
-## IP reputation
+### IP reputation
Mail originators are scored by multiple agencies about the quantity of traffic and spam they pass. This reputation is available via DNS queries to recipients as an additional signal. Sourceware has signed up with [https://www.dnswl.org/], a commonly used whitelist.
-## spam filtering
+### spam filtering
Receiver side tools including spamassassin collect validation signals from the above mechanisms, email content analysis, crowdsourced indications of spamminess, to arrive at a scalar numeric score. Email hitting a spam threshold is rejected, generally during the SMTP transaction itself.
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn
index 3ec5f83..c053a28 100644
--- a/TheCaseForFromRewriting.mdwn
+++ b/TheCaseForFromRewriting.mdwn
@@ -101,8 +101,9 @@ Yes, it is.
Yes, if you just "git am" received emails, the Author field of the git commit may be screwed up.
However, you can
- * hand-amend the author field of the commit ("git commit --amend --author=FOO")
- * instruct your contributors to send git patches with a safe From: author field in the patch body by adding this into their .gitconfig:
+
+* hand-amend the author field of the commit ("git commit --amend --author=FOO")
+* instruct your contributors to send git patches with a safe From: author field in the patch body by adding this into their .gitconfig:
[format]
from = true
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn
index e5e37bd..3ec5f83 100644
--- a/TheCaseForFromRewriting.mdwn
+++ b/TheCaseForFromRewriting.mdwn
@@ -101,8 +101,8 @@ Yes, it is.
Yes, if you just "git am" received emails, the Author field of the git commit may be screwed up.
However, you can
-* hand-amend the author field of the commit ("git commit --amend --author=FOO")
-* instruct your contributors to send git patches with a safe From: author field in the patch body by adding this into their .gitconfig:
+ * hand-amend the author field of the commit ("git commit --amend --author=FOO")
+ * instruct your contributors to send git patches with a safe From: author field in the patch body by adding this into their .gitconfig:
[format]
from = true
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn
index e2d0172..e5e37bd 100644
--- a/TheCaseForFromRewriting.mdwn
+++ b/TheCaseForFromRewriting.mdwn
@@ -90,7 +90,7 @@ The policy cure to this problem is to rewrite the From: field of the email as it
Reply-To: Overseers mailing list <overseers@sourceware.org>
Cc: Mark Wielaard <mark@klomp.org>, [...]
-The effect is that the user-visible "From:" address identifies the name of the person. But instead of his/her email address, the local mailing list name is used as email address, and the person's email is added to Cc: and/or Reply-To:. From a mail authentication perspective, this makes all the difference, as SPF, DKIM, etc., all treat this new message as though it was locally originated, and cryptographically authenticate it with DKIM etc. Receivers validate and accept it. People can read it.
+The effect is that the user-visible "From:" address identifies the name of the person. But instead of his/her email address, the local mailing list name is used as email address, and the person's email is added to Cc: and/or Reply-To:. From a mail authentication perspective, this makes all the difference, as SPF, DKIM, etc., all treat this new message as though it was locally originated, and cryptographically authenticate it with DKIM etc. Receivers validate and accept it. People can read it. People can reply to it.
## But it's ugly to look at.
@@ -99,6 +99,7 @@ Yes, it is.
## And it breaks git patches.
Yes, if you just "git am" received emails, the Author field of the git commit may be screwed up.
+
However, you can
* hand-amend the author field of the commit ("git commit --amend --author=FOO")
* instruct your contributors to send git patches with a safe From: author field in the patch body by adding this into their .gitconfig:
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn
index 953dfeb..e2d0172 100644
--- a/TheCaseForFromRewriting.mdwn
+++ b/TheCaseForFromRewriting.mdwn
@@ -85,11 +85,11 @@ There is an inherent conflict between forwarding email through mailing lists and
The DMARC policies aim to prove to email recipients that the email either arrived directly from its From:-visible author (SPF), or may have been relayed but was unmolested on the way (DKIM). The first criterion clearly cannot be met by a mailing list relay, which leaves DKIM as the only way to assure receiving systems. However, a DKIM signature associated with a given sender needs to come from a server associated with that author (with access to the domain's private keys), which a third party mailing list host cannot. Even if the author's originating mail server does sign the outgoing message with DKIM, and sourceware relays that signature, mailman could accidentally invalidate that signature by adding list-related headers that were not there before and were ["oversigned"](https://forum.dmarcian.com/t/oversigning-headers-what-is-it-and-how-does-one-do-it/2462). Some MTA software such as Exim defaults to oversigning several list-related fields. This results in invalid DKIM signatures, and thus a DMARC failure, and thus mail bouncing.
The policy cure to this problem is to rewrite the From: field of the email as it passes through the mailing list. This looks like:
-<blockquote>
-From: Mark Wielaard via Overseers <overseers@sourceware.org>
-Reply-To: Overseers mailing list <overseers@sourceware.org>
-Cc: Mark Wielaard <mark@klomp.org>, [...]
-</blockquote>
+
+ From: Mark Wielaard via Overseers <overseers@sourceware.org>
+ Reply-To: Overseers mailing list <overseers@sourceware.org>
+ Cc: Mark Wielaard <mark@klomp.org>, [...]
+
The effect is that the user-visible "From:" address identifies the name of the person. But instead of his/her email address, the local mailing list name is used as email address, and the person's email is added to Cc: and/or Reply-To:. From a mail authentication perspective, this makes all the difference, as SPF, DKIM, etc., all treat this new message as though it was locally originated, and cryptographically authenticate it with DKIM etc. Receivers validate and accept it. People can read it.
## But it's ugly to look at.
@@ -102,8 +102,8 @@ Yes, if you just "git am" received emails, the Author field of the git commit ma
However, you can
* hand-amend the author field of the commit ("git commit --amend --author=FOO")
* instruct your contributors to send git patches with a safe From: author field in the patch body by adding this into their .gitconfig:
-<code>
-[format]
- from = true
- forceInBodyFrom = true
-</code>
+
+ [format]
+ from = true
+ forceInBodyFrom = true
+
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn index 2a52b16..953dfeb 100644 --- a/TheCaseForFromRewriting.mdwn +++ b/TheCaseForFromRewriting.mdwn @@ -2,7 +2,6 @@ Email used to be easy: receive it, deliver it. Then it became harder: receive i [[!toc]] - # Technology background: email authentication technologies ## [SPF](https://en.wikipedia.org/wiki/Sender_Policy_Framework)
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn
index b3355d8..2a52b16 100644
--- a/TheCaseForFromRewriting.mdwn
+++ b/TheCaseForFromRewriting.mdwn
@@ -1,4 +1,4 @@
-Email used to be easy: receive it, deliver it. Then it became harder: receive it, spamassassin it, bounce or deliver it. But now, with the firehose of spam targeting billions of mail accounts, mail services have had to adapt increasingly draconian measures to filter out spam, spoofs, but reliably deliver real mail.
+Email used to be easy: receive it, deliver it. Then it became harder: receive it, spamassassin it, bounce or deliver it. But now, with the firehose of spam targeting billions of mail accounts, mail services have had to adapt increasingly draconian measures to filter out spam and forgery but reliably deliver real mail.
[[!toc]]
@@ -23,9 +23,9 @@ Receiver: verifies each DKIM signature against received subsequent headers+body
## [ARC](https://en.wikipedia.org/wiki/Authenticated_Received_Chain)
-Purpose: to verify that intermediate servers passed along headers, including DKIM data.
+Purpose: to verify that intermediate servers passed along authentication results.
-Sender: adds ARC headers for emails passing through a system (such as a mailing list).
+Sender: adds ARC headers for authenticated emails traveling through system.
Receiver: checks ARC signatures.
@@ -87,5 +87,24 @@ The DMARC policies aim to prove to email recipients that the email either arrive
The policy cure to this problem is to rewrite the From: field of the email as it passes through the mailing list. This looks like:
<blockquote>
-
+From: Mark Wielaard via Overseers <overseers@sourceware.org>
+Reply-To: Overseers mailing list <overseers@sourceware.org>
+Cc: Mark Wielaard <mark@klomp.org>, [...]
</blockquote>
+The effect is that the user-visible "From:" address identifies the name of the person. But instead of his/her email address, the local mailing list name is used as email address, and the person's email is added to Cc: and/or Reply-To:. From a mail authentication perspective, this makes all the difference, as SPF, DKIM, etc., all treat this new message as though it was locally originated, and cryptographically authenticate it with DKIM etc. Receivers validate and accept it. People can read it.
+
+## But it's ugly to look at.
+
+Yes, it is.
+
+## And it breaks git patches.
+
+Yes, if you just "git am" received emails, the Author field of the git commit may be screwed up.
+However, you can
+* hand-amend the author field of the commit ("git commit --amend --author=FOO")
+* instruct your contributors to send git patches with a safe From: author field in the patch body by adding this into their .gitconfig:
+<code>
+[format]
+ from = true
+ forceInBodyFrom = true
+</code>
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn index 2d20601..b3355d8 100644 --- a/TheCaseForFromRewriting.mdwn +++ b/TheCaseForFromRewriting.mdwn @@ -83,3 +83,9 @@ There is an inherent conflict between forwarding email through mailing lists and # The controversy: mailing list From: rewriting +The DMARC policies aim to prove to email recipients that the email either arrived directly from its From:-visible author (SPF), or may have been relayed but was unmolested on the way (DKIM). The first criterion clearly cannot be met by a mailing list relay, which leaves DKIM as the only way to assure receiving systems. However, a DKIM signature associated with a given sender needs to come from a server associated with that author (with access to the domain's private keys), which a third party mailing list host cannot. Even if the author's originating mail server does sign the outgoing message with DKIM, and sourceware relays that signature, mailman could accidentally invalidate that signature by adding list-related headers that were not there before and were ["oversigned"](https://forum.dmarcian.com/t/oversigning-headers-what-is-it-and-how-does-one-do-it/2462). Some MTA software such as Exim defaults to oversigning several list-related fields. This results in invalid DKIM signatures, and thus a DMARC failure, and thus mail bouncing. + +The policy cure to this problem is to rewrite the From: field of the email as it passes through the mailing list. This looks like: +<blockquote> + +</blockquote>
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn index 1789fef..2d20601 100644 --- a/TheCaseForFromRewriting.mdwn +++ b/TheCaseForFromRewriting.mdwn @@ -1,5 +1,8 @@ Email used to be easy: receive it, deliver it. Then it became harder: receive it, spamassassin it, bounce or deliver it. But now, with the firehose of spam targeting billions of mail accounts, mail services have had to adapt increasingly draconian measures to filter out spam, spoofs, but reliably deliver real mail. +[[!toc]] + + # Technology background: email authentication technologies ## [SPF](https://en.wikipedia.org/wiki/Sender_Policy_Framework)
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn
index 04f0be4..1789fef 100644
--- a/TheCaseForFromRewriting.mdwn
+++ b/TheCaseForFromRewriting.mdwn
@@ -1,6 +1,6 @@
-Email used to be easy: receive it, deliver it. Then it became harder: receive it, spamassassin it, bounce or deliver it. But now, with the firehose of spam targeting billions of mail accounts, mail services have had to adapt increasingly draconian measures to filter out spam but reliably deliver real mail.
+Email used to be easy: receive it, deliver it. Then it became harder: receive it, spamassassin it, bounce or deliver it. But now, with the firehose of spam targeting billions of mail accounts, mail services have had to adapt increasingly draconian measures to filter out spam, spoofs, but reliably deliver real mail.
-Among these measures:
+# Technology background: email authentication technologies
## [SPF](https://en.wikipedia.org/wiki/Sender_Policy_Framework)
@@ -28,7 +28,11 @@ Receiver: checks ARC signatures.
## [SRS](https://en.wikipedia.org/wiki/Sender_Rewriting_Scheme)
-Purpose: allow email forwarding
+Purpose: allow forwarded emails to trigger bounces that go back to originator
+
+Sender: rewrites the message envelope with a temporarily valid address that's associated with the originator
+
+Receiver: replies to temporary address if an email is rejected/bounced
## [DMARC](https://en.wikipedia.org/wiki/DMARC)
@@ -38,6 +42,41 @@ Sender: publishes a policy record in DNS
Receiver: verifies that message was signed with DKIM and/or came directly from an approved sender for the user-visible ("From:") domain.
+## personal email cryptography
+
+Additional cryptography layers added by the end-users via PGP or S/MIME but these do not really help signal their authenticity. Personal signatures are hard to verify (key distribution problem). Encryption frustrates content based analysis.
+
+## IP reputation
+
+Mail originators are scored by multiple agencies about the quantity of traffic and spam they pass. This reputation is available via DNS queries to recipients as an additional signal. Sourceware has signed up with [https://www.dnswl.org/], a commonly used whitelist.
+
+## spam filtering
+
+Receiver side tools including spamassassin collect validation signals from the above mechanisms, email content analysis, crowdsourced indications of spamminess, to arrive at a scalar numeric score. Email hitting a spam threshold is rejected, generally during the SMTP transaction itself.
+
# Policies of major mail servers
-Yahoo, google, and microsoft
+Yahoo, google, and microsoft are increasingly demanding DMARC alignment
+
+Sourceware's postfix mail server includes "milter" plugins for all of the above mail server tech.
+
+## Incoming mail
+
+Incoming mail is checked against all the criteria above, and positive and negative signals mixed into local spamassassin calculations. Messages that pass the criteria are delivered to user, service, or mailing list accounts as appropriate.
+
+## Outgoing mail, locally originated
+
+Outgoing mail originating from sourceware services such as bugzilla is fully signed, protected, sealed, and is bound to be delivered. Our IP reputation appears to be fine.
+
+## Outgoing mail, personally forwarded
+
+Outgoing mail originating from forwarding by ordinary users, are subjected to SRS envelope rewriting.
+
+## Mailing list traffic, forwarded through postfix
+
+Mailing list traffic necessitates changes to normal email. A single email to a single email alias turns into many outgoing emails to many recipients, with a number of customary changes such as added headers to track mailing list links. Most importantly, an unsubscribe link! Each mailing list has distinct lists of subscribers, and distinct policies with respect to content and header adjustment operations.
+
+There is an inherent conflict between forwarding email through mailing lists and assuring authenticity of the original mail. The forwarded email will appear to come from a different sending machine and going to a different destination than original. Despite these changes, mailing list hosts must try to make the forwarded outgoing messages as deliverable as possible. Undeliverable mail is a double problem: not only will a subscriber miss a message, but enough undeliverable bounces will cause subscribers to be automatically removed from mailing lists.
+
+# The controversy: mailing list From: rewriting
+
diff --git a/TheCaseForFromRewriting.mdwn b/TheCaseForFromRewriting.mdwn
new file mode 100644
index 0000000..04f0be4
--- /dev/null
+++ b/TheCaseForFromRewriting.mdwn
@@ -0,0 +1,43 @@
+Email used to be easy: receive it, deliver it. Then it became harder: receive it, spamassassin it, bounce or deliver it. But now, with the firehose of spam targeting billions of mail accounts, mail services have had to adapt increasingly draconian measures to filter out spam but reliably deliver real mail.
+
+Among these measures:
+
+## [SPF](https://en.wikipedia.org/wiki/Sender_Policy_Framework)
+
+Purpose: to verify that email for a domain is directly originating from an IP address approved by that domain.
+
+Sender: lists names of outgoing mail servers in text records in DNS.
+
+Receiver: verifies envelope "From " header during SMTP connection, to check IP address vs. SPF records.
+
+## [DKIM](https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail)
+
+Purpose: to verify that portions of a message/body have not been altered by intermediaries.
+
+Sender: posts DKIM public keys for domain in DNS. Cryptographically signs a subset or superset of headers+body, and adds that signature as new header.
+
+Receiver: verifies each DKIM signature against received subsequent headers+body
+
+## [ARC](https://en.wikipedia.org/wiki/Authenticated_Received_Chain)
+
+Purpose: to verify that intermediate servers passed along headers, including DKIM data.
+
+Sender: adds ARC headers for emails passing through a system (such as a mailing list).
+
+Receiver: checks ARC signatures.
+
+## [SRS](https://en.wikipedia.org/wiki/Sender_Rewriting_Scheme)
+
+Purpose: allow email forwarding
+
+## [DMARC](https://en.wikipedia.org/wiki/DMARC)
+
+Purpose: to verify alignment of *user-visible* email headers & content with SPF and/or DKIM authentication.
+
+Sender: publishes a policy record in DNS
+
+Receiver: verifies that message was signed with DKIM and/or came directly from an approved sender for the user-visible ("From:") domain.
+
+# Policies of major mail servers
+
+Yahoo, google, and microsoft
diff --git a/index.mdwn b/index.mdwn
index 87e3060..085e09c 100644
--- a/index.mdwn
+++ b/index.mdwn
@@ -6,7 +6,7 @@
* [request form](https://sourceware.org/cgi-bin/pdw/ps_form.cgi)
* [self-service ssh key / email address reset](https://sourceware.org/sourceware/accountinfo.html)
* ssh key issues
-* email issues
+* email issues, [[TheCaseForFromRewriting]]
#### information for maintainers of hosted projects
Add angle brackets around URLs
diff --git a/servers-and-services-2026.mdwn b/servers-and-services-2026.mdwn index 32bec76..5d63c87 100644 --- a/servers-and-services-2026.mdwn +++ b/servers-and-services-2026.mdwn @@ -17,7 +17,7 @@ The vms in use are: * forge-stage (forgejo) on vm08 (2 vcores, 4G mem, 40G disk) * debuginfod (elfutils debuginfod) on vm07 (2 vcores, 8G mem, 50GB disk). -Future hypothetical VMs: +Future vms: * inbox (public-inbox, 6 vcores, 16MB mem, 160GB disk) * patchwork (patchwork, 4 vcores, 12GB mem, 30GB disk) @@ -34,7 +34,7 @@ With additional vms possible for cygwin, gitweb/cgit, bugzilla, dwarfstd and val * sourceware-builder3, 112 cores, 754Gi mem, 465G raid1 disk, 2*932G nvme -This has been partitioned into 2 buildbot workers for builder.sourceware.org and 2 forgejo runners for forge.sourceware.org +This has been partitioned into 2 buildbot workers for <https://builder.sourceware.org> and 2 forgejo runners for <https://forge.sourceware.org> * sw3bb1: 40 vcpus, 256GB mem, 500GB disk (Fedora Core OS, containers) * sw3bb2: 16 vcpus, 96GB mem, 420GB disk (Fedora Core OS, containers) @@ -51,8 +51,8 @@ This has been partitioned into 2 buildbot workers for builder.sourceware.org and The following servers and services have been donated by companies, organizations or individuals but are not administrated directly by Sourceware overseers. -* Linaro CI https://ci.linaro.org does CI builds against patchwork and forge merge requests (Linaro, Maxim Kuvyrkov, https://gitlab.com/LinaroLtd/tcwg/ci/) -* redhat-pt-bot does try builds against patchwork (Red Hat, DJ Delorie, https://gitlab.com/djdelorie/glibc-cicd) +* Linaro CI <https://ci.linaro.org> does CI builds against patchwork and forge merge requests (Linaro, Maxim Kuvyrkov, <https://gitlab.com/LinaroLtd/tcwg/ci/>) +* redhat-pt-bot does try builds against patchwork (Red Hat, DJ Delorie, <https://gitlab.com/djdelorie/glibc-cicd>) * fedora-ppc64le buildbot worker (Brno University, Dan Horák) * fedora-s390x buildbot worker (Marist University, Dan Horák) * debian-ppc64[be] buildbot worker (Thomas Fitzsimmons)
diff --git a/servers-and-services-2026.mdwn b/servers-and-services-2026.mdwn index 5c01c10..32bec76 100644 --- a/servers-and-services-2026.mdwn +++ b/servers-and-services-2026.mdwn @@ -17,7 +17,7 @@ The vms in use are: * forge-stage (forgejo) on vm08 (2 vcores, 4G mem, 40G disk) * debuginfod (elfutils debuginfod) on vm07 (2 vcores, 8G mem, 50GB disk). -Other vms reserved for: +Future hypothetical VMs: * inbox (public-inbox, 6 vcores, 16MB mem, 160GB disk) * patchwork (patchwork, 4 vcores, 12GB mem, 30GB disk)
Add other servers and services
diff --git a/servers-and-services-2026.mdwn b/servers-and-services-2026.mdwn index f590cf6..5c01c10 100644 --- a/servers-and-services-2026.mdwn +++ b/servers-and-services-2026.mdwn @@ -1,6 +1,6 @@ # Sourceware Servers and Services 2026 -End of 2025, start of 2026, Sourceware servers were moved into new datacenters from Red Hat (RDU3) and OSUOSL (SDC). All servers are VM-first now and there are no public services running on bare metal anymore. We also take advantage of the RH OSPO cloud (RDU3) and OSUOSL x86_64 and arm64 openstack setups (SDC) for extra compute. +End of 2025, start of 2026, Sourceware servers were moved into new datacenters from Red Hat (RDU3) and OSUOSL (SDC). All servers are VM-first now and there are no public services running on bare metal anymore. We also take advantage of the RH OSPO cloud (RDU3) and OSUOSL x86_64 and arm64 openstack setups (SDC) for extra compute, redundancy and backups. ## Bare-metal servers in RH RDU3 (running vms) @@ -22,7 +22,7 @@ Other vms reserved for: * inbox (public-inbox, 6 vcores, 16MB mem, 160GB disk) * patchwork (patchwork, 4 vcores, 12GB mem, 30GB disk) * buildbot (buildbot, 4 vcores, 32GB mem, 48GB disk) -* bunsen (8 vcores, 12GB mem, 500GB disk). +* bunsen (bunsen, 8 vcores, 12GB mem, 500GB disk). With additional vms possible for cygwin, gitweb/cgit, bugzilla, dwarfstd and valgrind. @@ -46,3 +46,19 @@ This has been partitioned into 2 buildbot workers for builder.sourceware.org and * snapshots, 4 vcores, 8Gi mem, 620G vdisk (x86_64, alma9) * osuosl-arm64, 8 vcores, 16Gi mem, 160G vdisk (arm64, fedora) * osuosl-arm64-2, 16 vcores, 32Gi mem, 320G vdisk (arm64, debian) + +## Other servers and services + +The following servers and services have been donated by companies, organizations or individuals but are not administrated directly by Sourceware overseers. + +* Linaro CI https://ci.linaro.org does CI builds against patchwork and forge merge requests (Linaro, Maxim Kuvyrkov, https://gitlab.com/LinaroLtd/tcwg/ci/) +* redhat-pt-bot does try builds against patchwork (Red Hat, DJ Delorie, https://gitlab.com/djdelorie/glibc-cicd) +* fedora-ppc64le buildbot worker (Brno University, Dan Horák) +* fedora-s390x buildbot worker (Marist University, Dan Horák) +* debian-ppc64[be] buildbot worker (Thomas Fitzsimmons) +* debian-i386 and debian-armhf buildbot workers (Mark Wielaard) +* fedrawhide-x86_64 buildbot worker (Frank Eigler) +* ibm_power9 and ibm_power10 buildbot workers (IBM, Carl Love) +* starfive-[1-4] buildbot workers, VisionFive-2 risc-v boards (donated by Starfive, run by Mark Wielaard) +* Milk-V Pioneer Box donated by RISC-V International and SOPHGO (decommissioned) +* stap-*-x86_64 buildbot workers (Red Hat, Serhei Makarov)
List of servers, VMs and services
diff --git a/servers-and-services-2026.mdwn b/servers-and-services-2026.mdwn new file mode 100644 index 0000000..f590cf6 --- /dev/null +++ b/servers-and-services-2026.mdwn @@ -0,0 +1,48 @@ +# Sourceware Servers and Services 2026 + +End of 2025, start of 2026, Sourceware servers were moved into new datacenters from Red Hat (RDU3) and OSUOSL (SDC). All servers are VM-first now and there are no public services running on bare metal anymore. We also take advantage of the RH OSPO cloud (RDU3) and OSUOSL x86_64 and arm64 openstack setups (SDC) for extra compute. + +## Bare-metal servers in RH RDU3 (running vms) + +* server1, 112 cores, 1.5Ti mem, 14T raid6 disk +* server2, 64 cores, 500Gi mem, 4.4T raid6 disk +* server3, 64 cores, 188Gi mem, 4.4T raid6 disk + +These servers aren't directly accessible, but do provide 8 public ips throughout various vms (vm01 to vm08) that are publicly accessible. + +The vms in use are: + +* core services (dns, httpd, ssh, git) on vm01 (56 vcores, 500G mem, 4TB disk) +* forge (forgejo) on vm02 (16 vcores, 48G mem, 200G disk) +* forge-stage (forgejo) on vm08 (2 vcores, 4G mem, 40G disk) +* debuginfod (elfutils debuginfod) on vm07 (2 vcores, 8G mem, 50GB disk). + +Other vms reserved for: + +* inbox (public-inbox, 6 vcores, 16MB mem, 160GB disk) +* patchwork (patchwork, 4 vcores, 12GB mem, 30GB disk) +* buildbot (buildbot, 4 vcores, 32GB mem, 48GB disk) +* bunsen (8 vcores, 12GB mem, 500GB disk). + +With additional vms possible for cygwin, gitweb/cgit, bugzilla, dwarfstd and valgrind. + +## RH OSPO cloud server + +* rh-ospo-sourceware01, 3 vcores, 6Gi mem, 8G+60G vdisks (old forge, now backup) + +## Bare-metal server in OSUOSL SDC (running vms) + +* sourceware-builder3, 112 cores, 754Gi mem, 465G raid1 disk, 2*932G nvme + +This has been partitioned into 2 buildbot workers for builder.sourceware.org and 2 forgejo runners for forge.sourceware.org + +* sw3bb1: 40 vcpus, 256GB mem, 500GB disk (Fedora Core OS, containers) +* sw3bb2: 16 vcpus, 96GB mem, 420GB disk (Fedora Core OS, containers) +* sw3runner1: 40 vcpus, 256GB mem, 500GB disk (Debian, Forgejo Runner) +* sw3runner2: 16 vcpus, 96GB mem, 420GB disk (Debian, Forgejo Runner) + +## OSL cloud servers + +* snapshots, 4 vcores, 8Gi mem, 620G vdisk (x86_64, alma9) +* osuosl-arm64, 8 vcores, 16Gi mem, 160G vdisk (arm64, fedora) +* osuosl-arm64-2, 16 vcores, 32Gi mem, 320G vdisk (arm64, debian)
Add note about last step always needing to be done.
diff --git a/EmailNewMailmanList.mdwn b/EmailNewMailmanList.mdwn index 064135e..b1d0e20 100644 --- a/EmailNewMailmanList.mdwn +++ b/EmailNewMailmanList.mdwn @@ -22,7 +22,8 @@ Note, listnames need to be unique between the domains. * Coordinate the list moderator and password -* Set EmailHeaderRewriting as requested. +* Set [[EmailHeaderRewriting]] as requested. -* As root cd /etc/mailman/ && mailman/aliases-inbox; check result with git diff +* As root cd /etc/mailman/ && ./mailman-aliases-to-inbox.sh; check result with git diff +Note that last step needs to be done even if there are no inbox archives.
Adding an mailman mailinglist plus public-inbox archive.
diff --git a/EmailNewMailmanList.mdwn b/EmailNewMailmanList.mdwn new file mode 100644 index 0000000..064135e --- /dev/null +++ b/EmailNewMailmanList.mdwn @@ -0,0 +1,28 @@ +There are different mailman email domains sourceware.org, cygwin.com, gcc.gnu.org and lists.dwarfstd.org. +lists.dwarfstd.org isn't currently archived at inbox.sourceware.org, so you can skip the inbox setup instructions for those. +Note, listnames need to be unique between the domains. + +* First initialize the public-inbox archive. + + public-inbox-init -V 2 -L full --ng LISTNAME.[sourceware|gcc|cygwin].LISTNAME LISTNAME /home/inbox/lists/LISTNAME https://inbox.sourceware.org/LISTNAME LISTNAME@[sourceware.org|gcc.gnu.org|cygwin.org] + +* Make sure the .public-inbox/config files contains an entry for the listid + + [publicinbox "LISTNAME"] + address = LISTNAME@[sourceware.org|gcc.gnu.org|cygwin.com] + url = https://inbox.sourceware.org/LISTNAME + inboxdir = /home/inbox/lists/LISNAME + indexlevel = full + newsgroup = inbox.[sourceware|gcc|cygwin].bpf + listid = LISTNAME.[sourceware.org|gcc.gnu.org|cygwin.com] + +* As the mailman user create the list: + + bin/newlist --urlhost=[sourceware.org|gcc.gnu.org|cygwin.com|lists.dwarfstd.org] --emailhost=[sourceware.org|gcc.gnu.org|cygwin.com|lists.dwarfstd.org] LISTNAME + +* Coordinate the list moderator and password + +* Set EmailHeaderRewriting as requested. + +* As root cd /etc/mailman/ && mailman/aliases-inbox; check result with git diff +
diff --git a/backup.mdwn b/backup.mdwn new file mode 100644 index 0000000..b890220 --- /dev/null +++ b/backup.mdwn @@ -0,0 +1,13 @@ +Since [[Migration2025]], sourceware runs on a collection of VMs on a small number of bare-metal servers. This allows multiple backup strategies: + +* "none" :-), meaning, the crown jewels of the sourceware forges are the git repos. These repos already exist duplicated on hundreds of developer workstations and major services. These can serve as the germs of recovery from a total disaster. + +* public rsync. Mailing list archives and some other resources are actively mirrored via our rsync services. Ditto partial, but useful for disaster recovery. + +* local git-backed config files. Cron jobs on sourceware git-control a variety of config files and directories for informal configuration management. See root crontab(1). + +* remote system-wide rsync. Frank operates a periodic far-off-site rsync of the sourceware.org primary server filesystems, including all of the above, plus bugzilla databases etc. + +* VM host LVM snapshots. We have started rolling out periodic automated LVM snapshots of the locally hosted VMs that make up the bulk of sourceware servers. The host machines are not directly accessible from the internet. See <code>/root/VM-BACKUP.sh</code>, a simple little shell script, and usual LVM commands. + +* future: cross-host VM LVM snapshots. Once we have all our RDU2 and RDU3 hardware together again in RDU3, we can warm-backup VM LVM snapshots from the primary server to another one that's beefy enough to run sourceware. That should allow near-instant switchover in the case of severe hardware failure.
diff --git a/index.mdwn b/index.mdwn
index e745c67..87e3060 100644
--- a/index.mdwn
+++ b/index.mdwn
@@ -24,9 +24,7 @@
* reach them by IRC at [ircs://irc.libera.chat:6697](https://web.libera.chat/#overseers)
* [[who are the overseers|OSStaff]]
-* system architecture
- * server2.sourceware.org = gcc.gnu.org
- * server3.sourceware.org - warm backup
+* [[system architecture]]
* project hosting policies
* account policies
* [[aging]]
@@ -54,7 +52,7 @@
* mysql
* gerrit
* recipes:
- * creating new [[user|OSNewUser]], [[project|OSNewProject]], [[git repo|OSNewGitRepo]], [[mailing list|OSNewMailingList]], [[web hosting site|OSNewWebHosting]], [[moinmoin wiki|OSNewMoinWiki]], bugzilla, [[patchwork]]
+ * creating new [[user|OSNewUser]], [[project|OSNewProject]], [[git repo|OSNewGitRepo]], [[mailing list|OSNewMailingList]], [[web hosting site|OSNewWebHosting]], [[moinmoin wiki|OSNewMoinWiki]], bugzilla, [[patchwork]], [[backup]]
#### Migration
dmarc_moderation_action (privacy,sender_filters):
diff --git a/EmailHeaderRewriting.mdwn b/EmailHeaderRewriting.mdwn index 678f0c7..0b120d6 100644 --- a/EmailHeaderRewriting.mdwn +++ b/EmailHeaderRewriting.mdwn @@ -15,7 +15,7 @@ Which means the following settings: * msg_header (nondigest): (empty) * msg_footer (nondigest): (empty) * scrub_nondigest (nondigest): No -* dmarc_moderation_action (privacy): Accept +* dmarc_moderation_action (privacy,sender_filters): Accept * filter_content (contentfilter): No Some more background can be found in this bug:
Add ref to bugzilla
diff --git a/EmailHeaderRewriting.mdwn b/EmailHeaderRewriting.mdwn index 75f599f..678f0c7 100644 --- a/EmailHeaderRewriting.mdwn +++ b/EmailHeaderRewriting.mdwn @@ -17,3 +17,7 @@ Which means the following settings: * scrub_nondigest (nondigest): No * dmarc_moderation_action (privacy): Accept * filter_content (contentfilter): No + +Some more background can be found in this bug: + +* https://sourceware.org/bugzilla/show_bug.cgi?id=29713
Add mailman admin settings for no From rewriting
diff --git a/EmailHeaderRewriting.mdwn b/EmailHeaderRewriting.mdwn new file mode 100644 index 0000000..75f599f --- /dev/null +++ b/EmailHeaderRewriting.mdwn @@ -0,0 +1,19 @@ +As a mailman list admin you can setup your list so it doesn't use From header rewriting (this implies no header rewriting at all, also no Reply-To mangling) + +To make this work we have to make sure none of the list settings change the email body, subject or any other header that is likely covered by a dkim signature. + +Which means the following settings: + +* subject_prefix (general): (empty) +* from_is_list (general): No +* anonymous_list (general): No +* first_strip_reply_to (general): No +* reply_goes_to_list (general): Poster +* reply_to_address (general): (empty) +* include_sender_header (general): No +* drop_cc (general): No +* msg_header (nondigest): (empty) +* msg_footer (nondigest): (empty) +* scrub_nondigest (nondigest): No +* dmarc_moderation_action (privacy): Accept +* filter_content (contentfilter): No
diff --git a/OSNewMailingList.mdwn b/OSNewMailingList.mdwn index 0279ce1..5daa0c3 100644 --- a/OSNewMailingList.mdwn +++ b/OSNewMailingList.mdwn @@ -1,6 +1,3 @@ -T - - NB: the following is obsolete in mailman/postfix days.
diff --git a/OSNewMailingList.mdwn b/OSNewMailingList.mdwn index 53ba06a..0279ce1 100644 --- a/OSNewMailingList.mdwn +++ b/OSNewMailingList.mdwn @@ -1,5 +1,12 @@ -These days there is a script for creating a new mailing list. The old -instructions [are here](https://sourceware.org/sourceware/procedures/new-list.txt), but using the script is much simpler. +T + + + +NB: the following is obsolete in mailman/postfix days. + +Back in the ezmlm/qmail days, there was a script for creating a new mailing list. + +The even older instructions [are here](https://sourceware.org/sourceware/procedures/new-list.txt), but using the script is much simpler. One must manually update <tt>https://sourceware.org/lists.html</tt> = <tt>/sourceware/www/sourceware/ml/lists.html</tt>. Also tweak <tt>/sourceware/www/sourceware/*/$project/$project.mrc</tt>. <pre>
More DNS details
diff --git a/OpenHouse2025.mdwn b/OpenHouse2025.mdwn
index 13a7030..f1de8c4 100644
--- a/OpenHouse2025.mdwn
+++ b/OpenHouse2025.mdwn
@@ -48,10 +48,41 @@ Discussing [[Migration2025]]
* Migration experiment, add the following to your /etc/hosts (make sure to take it out again!)
- 38.145.34.32 builder.sourceware.org
- 38.145.34.32 inbox.sourceware.org
- 38.145.34.32 patchwork.sourceware.org
38.145.34.32 sourceware.org
+ 38.145.34.32 gcc.gnu.org
+ 38.145.34.32 elfutils.org
+ 38.145.34.32 dwarfstd.org
+ 38.145.34.32 lists.dwarfstd.org
+ 38.145.34.32 valgrind.org
+ 38.145.34.32 cygwin.com
+ 38.145.34.32 cygwin.org
+ 38.145.34.32 cygwin.net
+ 38.145.34.32 www.cygwin.com
+ 38.145.34.32 www.cygwin.org
+ 38.145.34.32 www.cygwin.net
+ 38.145.34.32 ftp.cygwin.com
+ 38.145.34.32 ftp.cygwin.org
+ 38.145.34.32 ftp.cygwin.net
+
+ {www,builder,inbox,patchwork,ecos,gcc}.sourceware.org are all CNAMEs so don't need the (new) address.
+
+ forge.sourceware.org uses an RH OSCI VM
+
+ snapshots.sourceware.org uses a server at osuosl.
+
+ forge-stage.sourceware.org is already vm08 (doesn't use CNAME but A record for MX).
+
+ debuginfod.elfutils.org is already an alias for vm07
+
+ {www,git,wiki}.dwarfstd.org are CNAMEs for sourceware.org
+
+ www.valgrind.org is a CNAME for sourceware.org
+
+ systemtap.org is maintained by Sourceware PLC/Conservancy, but DNS is handled by elastic.org
+
+ Note {www,ftp}.cygwin.{com,org,net} are CNAMEs for server2.sourceware.org (so above they get an new address)
+
+ Also note that there are no IPv6 addresses available in RDU3 (yet?). And rDNS isn't mapped (yet?).
## Backups
Link OpenHouse2025
diff --git a/Migration2025.mdwn b/Migration2025.mdwn index de43ff3..1efb10b 100644 --- a/Migration2025.mdwn +++ b/Migration2025.mdwn @@ -55,3 +55,7 @@ Guiding principles: no direct file sharing between/across VMs. * ? * LVM snapshot copies from server1 + +# Other + +See also [[OpenHouse2025]]
Backups
diff --git a/OpenHouse2025.mdwn b/OpenHouse2025.mdwn
index a30900b..13a7030 100644
--- a/OpenHouse2025.mdwn
+++ b/OpenHouse2025.mdwn
@@ -53,3 +53,7 @@ Discussing [[Migration2025]]
38.145.34.32 patchwork.sourceware.org
38.145.34.32 sourceware.org
+## Backups
+
+ * Should be done through lvm snapshots. Which can then be rsynced to the "old" server2 or server3.
+ * Some debate what to do about databases, should they be put in "backup mode", dumped regularly as .sql files?
diff --git a/OpenHouse2025.mdwn b/OpenHouse2025.mdwn index 536195c..a30900b 100644 --- a/OpenHouse2025.mdwn +++ b/OpenHouse2025.mdwn @@ -48,8 +48,8 @@ Discussing [[Migration2025]] * Migration experiment, add the following to your /etc/hosts (make sure to take it out again!) - 38.145.34.32 builder.sourceware.org - 38.145.34.32 inbox.sourceware.org - 38.145.34.32 patchwork.sourceware.org - 38.145.34.32 sourceware.org + 38.145.34.32 builder.sourceware.org + 38.145.34.32 inbox.sourceware.org + 38.145.34.32 patchwork.sourceware.org + 38.145.34.32 sourceware.org
Add network
diff --git a/OpenHouse2025.mdwn b/OpenHouse2025.mdwn
index 7d9391a..536195c 100644
--- a/OpenHouse2025.mdwn
+++ b/OpenHouse2025.mdwn
@@ -41,3 +41,15 @@ Discussing [[Migration2025]]
* forge (32GB mem, 8 vcpu, 12GB + 128GB lvs), as soon as forge-stage ansible setup works out for production usage
* bunsen (12GB mem, 8 vcpu, 500GB lvs)
* cygwin, gitweb/cgit, bugzilla, dwarfstd, valgrind
+
+## Network
+
+ * For network setup we have a /24 with 8 public addresses that can be staticly assigned to each VM (if we need more, we can get more).
+
+ * Migration experiment, add the following to your /etc/hosts (make sure to take it out again!)
+
+ 38.145.34.32 builder.sourceware.org
+ 38.145.34.32 inbox.sourceware.org
+ 38.145.34.32 patchwork.sourceware.org
+ 38.145.34.32 sourceware.org
+
Add services current/new versions
diff --git a/OpenHouse2025.mdwn b/OpenHouse2025.mdwn
index c7dd144..7d9391a 100644
--- a/OpenHouse2025.mdwn
+++ b/OpenHouse2025.mdwn
@@ -35,9 +35,9 @@ Discussing [[Migration2025]]
* debuginfod-elfutils (8GB mem, 2 vcpu, 50 GiB lv), [[https://debuginfod.elfutils.org]], already in production
* sourceware (256GB mem, 8 vcpu, 3.91 TiB lv), hosting everything from server2 which isn't already in a separate VM
* Other proposed VMs from [[Migration2025]]
- * inbox (16GB mem, 6 vcpu, 8GB + 120GB lvs)
- * patchwork (12GB mem, 4 vcpu, 12GB + 12GB lvs)
- * buildbot (32GB mem, 4 vcpu, 8GB + 30GB lvs)
- * forge (32GB mem, 8 vcpu, 12GB + 128GB lvs)
+ * inbox (16GB mem, 6 vcpu, 8GB + 120GB lvs), current version 1.9.0 (from epel), latest 2.0.0
+ * patchwork (12GB mem, 4 vcpu, 12GB + 12GB lvs), current version v3.1.1 [django 3.2.x] (in python env with local patches), latest v3.2.1 [django 5.2.x]
+ * buildbot (32GB mem, 4 vcpu, 8GB + 30GB lvs), current version 3.9.2 (python env with 1 local patch, see SETUP), latest 4.3.0
+ * forge (32GB mem, 8 vcpu, 12GB + 128GB lvs), as soon as forge-stage ansible setup works out for production usage
* bunsen (12GB mem, 8 vcpu, 500GB lvs)
- * cygwin
+ * cygwin, gitweb/cgit, bugzilla, dwarfstd, valgrind
URLs
diff --git a/OpenHouse2025.mdwn b/OpenHouse2025.mdwn
index ddac755..c7dd144 100644
--- a/OpenHouse2025.mdwn
+++ b/OpenHouse2025.mdwn
@@ -31,8 +31,8 @@ Discussing [[Migration2025]]
* Access is currently only available to fche and mjw. No direct public internet access. We could setup a jumphost.
* But probably ok to have a (read-only) git repo with just the libvirt configs
* Currently running just three VMs
- * forge-stage-deb13 (4GB mem, 2 vcpu, 20 GiB lv), [[forge-stage.sourceware.org]], ansible managed
- * debuginfod-elfutils (8GB mem, 2 vcpu, 50 GiB lv), [[debuginfod.elfutils.org]], already in production
+ * forge-stage-deb13 (4GB mem, 2 vcpu, 20 GiB lv), [[https://forge-stage.sourceware.org]], ansible managed
+ * debuginfod-elfutils (8GB mem, 2 vcpu, 50 GiB lv), [[https://debuginfod.elfutils.org]], already in production
* sourceware (256GB mem, 8 vcpu, 3.91 TiB lv), hosting everything from server2 which isn't already in a separate VM
* Other proposed VMs from [[Migration2025]]
* inbox (16GB mem, 6 vcpu, 8GB + 120GB lvs)
explain current running vms
diff --git a/OpenHouse2025.mdwn b/OpenHouse2025.mdwn
index 0d249c0..ddac755 100644
--- a/OpenHouse2025.mdwn
+++ b/OpenHouse2025.mdwn
@@ -31,9 +31,9 @@ Discussing [[Migration2025]]
* Access is currently only available to fche and mjw. No direct public internet access. We could setup a jumphost.
* But probably ok to have a (read-only) git repo with just the libvirt configs
* Currently running just three VMs
- * forge-stage-deb13 (4GB mem, 2 vcpu, 20 GiB lv)
- * debuginfod-elfutils (8GB mem, 2 vcpu, 50 GiB lv)
- * sourceware (256GB mem, 8 vcpu, 3.91 TiB lv)
+ * forge-stage-deb13 (4GB mem, 2 vcpu, 20 GiB lv), [[forge-stage.sourceware.org]], ansible managed
+ * debuginfod-elfutils (8GB mem, 2 vcpu, 50 GiB lv), [[debuginfod.elfutils.org]], already in production
+ * sourceware (256GB mem, 8 vcpu, 3.91 TiB lv), hosting everything from server2 which isn't already in a separate VM
* Other proposed VMs from [[Migration2025]]
* inbox (16GB mem, 6 vcpu, 8GB + 120GB lvs)
* patchwork (12GB mem, 4 vcpu, 12GB + 12GB lvs)
Add other proposed VMs
diff --git a/OpenHouse2025.mdwn b/OpenHouse2025.mdwn
index b806087..0d249c0 100644
--- a/OpenHouse2025.mdwn
+++ b/OpenHouse2025.mdwn
@@ -34,3 +34,10 @@ Discussing [[Migration2025]]
* forge-stage-deb13 (4GB mem, 2 vcpu, 20 GiB lv)
* debuginfod-elfutils (8GB mem, 2 vcpu, 50 GiB lv)
* sourceware (256GB mem, 8 vcpu, 3.91 TiB lv)
+ * Other proposed VMs from [[Migration2025]]
+ * inbox (16GB mem, 6 vcpu, 8GB + 120GB lvs)
+ * patchwork (12GB mem, 4 vcpu, 12GB + 12GB lvs)
+ * buildbot (32GB mem, 4 vcpu, 8GB + 30GB lvs)
+ * forge (32GB mem, 8 vcpu, 12GB + 128GB lvs)
+ * bunsen (12GB mem, 8 vcpu, 500GB lvs)
+ * cygwin
diff --git a/OpenHouse2025.mdwn b/OpenHouse2025.mdwn index af1ffd8..b806087 100644 --- a/OpenHouse2025.mdwn +++ b/OpenHouse2025.mdwn @@ -29,7 +29,8 @@ Discussing [[Migration2025]] * New server (already in RDU3) has been setup with a minimal RHEL 10 setup just for running VMs * Access is currently only available to fche and mjw. No direct public internet access. We could setup a jumphost. + * But probably ok to have a (read-only) git repo with just the libvirt configs * Currently running just three VMs - * forge-stage-deb13 - * debuginfod-elfutils - * sourceware + * forge-stage-deb13 (4GB mem, 2 vcpu, 20 GiB lv) + * debuginfod-elfutils (8GB mem, 2 vcpu, 50 GiB lv) + * sourceware (256GB mem, 8 vcpu, 3.91 TiB lv)
link webchat
diff --git a/OpenHouse2025.mdwn b/OpenHouse2025.mdwn index 7ca5d0f..af1ffd8 100644 --- a/OpenHouse2025.mdwn +++ b/OpenHouse2025.mdwn @@ -1,4 +1,5 @@ Our first full day Open House! Friday 14 2025, full day 09:00 to 17:00 UTC. +[[#overseers|https://web.libera.chat/#overseers]] on irc.libera.chat Discussing [[Migration2025]]
Add server1 VM-first setup
diff --git a/OpenHouse2025.mdwn b/OpenHouse2025.mdwn index f066e66..7ca5d0f 100644 --- a/OpenHouse2025.mdwn +++ b/OpenHouse2025.mdwn @@ -22,3 +22,13 @@ Discussing [[Migration2025]] * Sam sent a patch to always check signatures for source tars * [[https://inbox.sourceware.org/buildbot/168d47063b5286251d749a350b38939d23b77794.1763093965.git.sam@gentoo.org/]] * Discussion if/how we can trust [[https://ftp.gnu.org/gnu/gnu-keyring.gpg]] when using ftpmirror + * Pulled in glibc hackers to show how they can impove their own build-many-glibcs.py script + +# The new server1 VM-first setup + + * New server (already in RDU3) has been setup with a minimal RHEL 10 setup just for running VMs + * Access is currently only available to fche and mjw. No direct public internet access. We could setup a jumphost. + * Currently running just three VMs + * forge-stage-deb13 + * debuginfod-elfutils + * sourceware
diff --git a/OpenHouse2025.mdwn b/OpenHouse2025.mdwn
index 4ec80b3..f066e66 100644
--- a/OpenHouse2025.mdwn
+++ b/OpenHouse2025.mdwn
@@ -11,13 +11,14 @@ Discussing [[Migration2025]]
* New 2x28 cores (2x56 threads), 768GB RAM machine. Need to provide disks.
* Recommend Sourceware PLC to buy 2 x NVMe M.2 1TB (VMs/containers) and one 2.5" SATA drive of 500GB (base OS).
-* Security scans
- * Red Hat IT does scans of our servers and we discuss findings
- * We last [[updated|https://forge.sourceware.org/forge/forge/commit/0284f78640f8846332932a5956377b64fe125b5c]] the STARTLS crypto algos for forge.sourceware.org based on their recommendations
+# Security
-* Container package security checks
+* Red Hat IT does scans of our servers and we discuss findings
+* We last [[updated|https://forge.sourceware.org/forge/forge/commit/0284f78640f8846332932a5956377b64fe125b5c]] the STARTLS crypto algos for forge.sourceware.org based on their recommendations
+* Container package security checks
* builder.sourceware.org has various container builders. Some build gnu tools from source.
+ * [[https://sourceware.org/cgit/builder/tree/builder/containers]]
* Sam sent a patch to always check signatures for source tars
- * https://inbox.sourceware.org/buildbot/168d47063b5286251d749a350b38939d23b77794.1763093965.git.sam@gentoo.org/
- * Discussion if/how we can trust https://ftp.gnu.org/gnu/gnu-keyring.gpg when using ftpmirror
+ * [[https://inbox.sourceware.org/buildbot/168d47063b5286251d749a350b38939d23b77794.1763093965.git.sam@gentoo.org/]]
+ * Discussion if/how we can trust [[https://ftp.gnu.org/gnu/gnu-keyring.gpg]] when using ftpmirror
Container package security checks
diff --git a/OpenHouse2025.mdwn b/OpenHouse2025.mdwn index 53dccf8..4ec80b3 100644 --- a/OpenHouse2025.mdwn +++ b/OpenHouse2025.mdwn @@ -14,3 +14,10 @@ Discussing [[Migration2025]] * Security scans * Red Hat IT does scans of our servers and we discuss findings * We last [[updated|https://forge.sourceware.org/forge/forge/commit/0284f78640f8846332932a5956377b64fe125b5c]] the STARTLS crypto algos for forge.sourceware.org based on their recommendations + +* Container package security checks + + * builder.sourceware.org has various container builders. Some build gnu tools from source. + * Sam sent a patch to always check signatures for source tars + * https://inbox.sourceware.org/buildbot/168d47063b5286251d749a350b38939d23b77794.1763093965.git.sam@gentoo.org/ + * Discussion if/how we can trust https://ftp.gnu.org/gnu/gnu-keyring.gpg when using ftpmirror
Security scans
diff --git a/OpenHouse2025.mdwn b/OpenHouse2025.mdwn
index 92252f2..53dccf8 100644
--- a/OpenHouse2025.mdwn
+++ b/OpenHouse2025.mdwn
@@ -10,3 +10,7 @@ Discussing [[Migration2025]]
* Old 48G/62G mem, 260GB/450GB disk, 16/24 cores
* New 2x28 cores (2x56 threads), 768GB RAM machine. Need to provide disks.
* Recommend Sourceware PLC to buy 2 x NVMe M.2 1TB (VMs/containers) and one 2.5" SATA drive of 500GB (base OS).
+
+* Security scans
+ * Red Hat IT does scans of our servers and we discuss findings
+ * We last [[updated|https://forge.sourceware.org/forge/forge/commit/0284f78640f8846332932a5956377b64fe125b5c]] the STARTLS crypto algos for forge.sourceware.org based on their recommendations
Add OSUOSL build machine migration discussion
diff --git a/OpenHouse2025.mdwn b/OpenHouse2025.mdwn index 47a1b09..92252f2 100644 --- a/OpenHouse2025.mdwn +++ b/OpenHouse2025.mdwn @@ -1,2 +1,12 @@ Our first full day Open House! Friday 14 2025, full day 09:00 to 17:00 UTC. +Discussing [[Migration2025]] + +# Hardware resources + +* See [[Migration2025]]. + +* OSUOSL is offering to replace our current 2 x86_64 container builders with one bigger server. + * Old 48G/62G mem, 260GB/450GB disk, 16/24 cores + * New 2x28 cores (2x56 threads), 768GB RAM machine. Need to provide disks. + * Recommend Sourceware PLC to buy 2 x NVMe M.2 1TB (VMs/containers) and one 2.5" SATA drive of 500GB (base OS).
Create page
diff --git a/OpenHouse2025.mdwn b/OpenHouse2025.mdwn new file mode 100644 index 0000000..47a1b09 --- /dev/null +++ b/OpenHouse2025.mdwn @@ -0,0 +1,2 @@ +Our first full day Open House! Friday 14 2025, full day 09:00 to 17:00 UTC. +
diff --git a/index.mdwn b/index.mdwn index aca41a9..e745c67 100644 --- a/index.mdwn +++ b/index.mdwn @@ -65,7 +65,7 @@ Every second friday of the month in channel overseers on irc.libera.net at 16:00 UTC -* [[2025 Open House|OpenHouse2025] +* [[2025 Open House|OpenHouse2025]] <font size="-1">Archive of old documentation, from [[y2000 era|OldInfo2000]], from [[y2010 era|OldInfo2010]].</font>
Add openoffice/house
diff --git a/index.mdwn b/index.mdwn index 305176f..aca41a9 100644 --- a/index.mdwn +++ b/index.mdwn @@ -61,6 +61,12 @@ * [[2019 migration status|MigrationStatus]] [[2019 migration details|MigrationWorkItems]] * [[2025 migration plans|Migration2025]] +#### Open Office/House + +Every second friday of the month in channel overseers on irc.libera.net at 16:00 UTC + +* [[2025 Open House|OpenHouse2025] + <font size="-1">Archive of old documentation, from [[y2000 era|OldInfo2000]], from [[y2010 era|OldInfo2010]].</font> ---
diff --git a/Migration2025.mdwn b/Migration2025.mdwn
index 2eb0e34..de43ff3 100644
--- a/Migration2025.mdwn
+++ b/Migration2025.mdwn
@@ -7,7 +7,7 @@ Scratchpad for planning ideas for the winter-2025 migration.
* OSUOSL servers:
* 2 x86_64 smallish servers for container builders (48G/62G mem, 260GB/450GB disk, 16/24 cores)
* OSUOSL is offering to replace these two with one larger one. 2x28 cores (2x56 threads), 768GB RAM machine.
- We will have to provide disks ourselves. There are 2 x NVMe M.2 slots and 6 x 2.5" SATA drive bays.
+ Need to provide disks. There are 2 x NVMe M.2 slots and 6 x 2.5" SATA drive bays.
* snapshots server (8G mem, 620GB disk, 4 cores)
* 2 arm64 builders (16G/32G mem, 200GB/320GB disk, 8/16 cores).
* OSPO RH VM: forge.sourceware.org (6G mem, 8GB root /, 60G data /srv, 3 cores)
Add OSUOSL x86_64 michine migration/upgrade details
diff --git a/Migration2025.mdwn b/Migration2025.mdwn index 63dff76..2eb0e34 100644 --- a/Migration2025.mdwn +++ b/Migration2025.mdwn @@ -6,6 +6,8 @@ Scratchpad for planning ideas for the winter-2025 migration. * Old RH servers: server2.sourceware.org, server3.sourceware.org * OSUOSL servers: * 2 x86_64 smallish servers for container builders (48G/62G mem, 260GB/450GB disk, 16/24 cores) + * OSUOSL is offering to replace these two with one larger one. 2x28 cores (2x56 threads), 768GB RAM machine. + We will have to provide disks ourselves. There are 2 x NVMe M.2 slots and 6 x 2.5" SATA drive bays. * snapshots server (8G mem, 620GB disk, 4 cores) * 2 arm64 builders (16G/32G mem, 200GB/320GB disk, 8/16 cores). * OSPO RH VM: forge.sourceware.org (6G mem, 8GB root /, 60G data /srv, 3 cores)
Add more proposed VM (patchwork, buildbot) disk/mem/cores
diff --git a/Migration2025.mdwn b/Migration2025.mdwn index a4114bb..63dff76 100644 --- a/Migration2025.mdwn +++ b/Migration2025.mdwn @@ -23,11 +23,14 @@ Guiding principles: no direct file sharing between/across VMs. * sourceware.org + gcc.gnu.org: a RHEL8? megaVM containing most current sourceware stuff; git repos, bugzilla; mysql; gitweb; postfix; rsync * if practical: mailman: a RHEL8 VM running mailman2 + public-inbox; postfix; mailing-list archives by git-over-inbox/http only * We can upgrade this to something newer if we run mailmain2 in a container -* inbox: Provides http, imap, nntp, Needs (one) incoming email, 8GB main distro /, 120GB lists data /srv, 16GB mem, 6 cores. -* patchwork: +* inbox: Provides http, imap, nntp, Needs (one) incoming email + * Proposed VM: 8GB main distro /, 120GB lists data /srv, 16GB mem, 6 cores. +* patchwork: Needsits own database (or access to a database VM?), (one) incoming email + * Proposed VM (including own database): 12GB main distro OS /, 12GB patchwork data (2 GB patchwork + ?? database), 12GB mem, 4 cores * buildbot: (pushing bunsen data to sourceware:/git/bunsendb.git via cross-vm credential) + * Proposed VM: 8GB main distro OS /, 30GB data /srv (14G gitpoller, + 16G sqlite state), 32GB mem, 4 cores * forge: maybe repatriate this from RH OSPO hosting service in entirety - * 12GB main distro OS /, 128GB data /srv (120GB forgejo repos, 8GB postgresql), 32GB mem, 8 cores + * Proposed VM: 12GB main distro OS /, 128GB data /srv (120GB forgejo repos, 8GB postgresql), 32GB mem, 8 cores * forge-stage: a test environment for the above #### additional possible VMs:
diff --git a/Migration2025.mdwn b/Migration2025.mdwn index 847c808..a4114bb 100644 --- a/Migration2025.mdwn +++ b/Migration2025.mdwn @@ -8,7 +8,7 @@ Scratchpad for planning ideas for the winter-2025 migration. * 2 x86_64 smallish servers for container builders (48G/62G mem, 260GB/450GB disk, 16/24 cores) * snapshots server (8G mem, 620GB disk, 4 cores) * 2 arm64 builders (16G/32G mem, 200GB/320GB disk, 8/16 cores). -* OSPO RH VM: forge.sourceware.org (6G mem, 8GB root /, 60G ./srv, 3 cores) +* OSPO RH VM: forge.sourceware.org (6G mem, 8GB root /, 60G data /srv, 3 cores) # Software
Add forge.sourceware.org VM details
diff --git a/Migration2025.mdwn b/Migration2025.mdwn index d0437d9..847c808 100644 --- a/Migration2025.mdwn +++ b/Migration2025.mdwn @@ -8,6 +8,7 @@ Scratchpad for planning ideas for the winter-2025 migration. * 2 x86_64 smallish servers for container builders (48G/62G mem, 260GB/450GB disk, 16/24 cores) * snapshots server (8G mem, 620GB disk, 4 cores) * 2 arm64 builders (16G/32G mem, 200GB/320GB disk, 8/16 cores). +* OSPO RH VM: forge.sourceware.org (6G mem, 8GB root /, 60G ./srv, 3 cores) # Software
diff --git a/Migration2025.mdwn b/Migration2025.mdwn index 8afe20e..d0437d9 100644 --- a/Migration2025.mdwn +++ b/Migration2025.mdwn @@ -32,7 +32,7 @@ Guiding principles: no direct file sharing between/across VMs. #### additional possible VMs: * bunsen: cloning bunsen data from sourceware:/git/bunsendb.git; running indexer & web-ui - Disk need ~140GB+ for bunsendb git and ~300GB+ for bunsenweb and ~45GB+ for bunsenql (~485GB+ total data). + * Disk need ~140GB+ for bunsendb git and ~300GB+ for bunsenweb and ~45GB+ for bunsenql (~485GB+ total data). * gitweb/cgit; cloning git repos from sourceware:/git/ * moinmoin: wikis * cygwin?
Add inbox as VM with provides and needs
diff --git a/Migration2025.mdwn b/Migration2025.mdwn index 9ccefae..8afe20e 100644 --- a/Migration2025.mdwn +++ b/Migration2025.mdwn @@ -20,8 +20,9 @@ We wish to maintain appearance of current operation model (maintainers can direc Guiding principles: no direct file sharing between/across VMs. * sourceware.org + gcc.gnu.org: a RHEL8? megaVM containing most current sourceware stuff; git repos, bugzilla; mysql; gitweb; postfix; rsync -* if practical: mailman: a RHEL8 VM running mailman2 + public-inbox; postfix; mailing-list archives by git-over-inbox/http only + * if practical: mailman: a RHEL8 VM running mailman2 + public-inbox; postfix; mailing-list archives by git-over-inbox/http only * We can upgrade this to something newer if we run mailmain2 in a container +* inbox: Provides http, imap, nntp, Needs (one) incoming email, 8GB main distro /, 120GB lists data /srv, 16GB mem, 6 cores. * patchwork: * buildbot: (pushing bunsen data to sourceware:/git/bunsendb.git via cross-vm credential) * forge: maybe repatriate this from RH OSPO hosting service in entirety
Add some estimates for forge disk/mem/core needs
diff --git a/Migration2025.mdwn b/Migration2025.mdwn index 70bade4..9ccefae 100644 --- a/Migration2025.mdwn +++ b/Migration2025.mdwn @@ -25,6 +25,7 @@ Guiding principles: no direct file sharing between/across VMs. * patchwork: * buildbot: (pushing bunsen data to sourceware:/git/bunsendb.git via cross-vm credential) * forge: maybe repatriate this from RH OSPO hosting service in entirety + * 12GB main distro OS /, 128GB data /srv (120GB forgejo repos, 8GB postgresql), 32GB mem, 8 cores * forge-stage: a test environment for the above #### additional possible VMs:
diff --git a/Migration2025.mdwn b/Migration2025.mdwn index 4937779..70bade4 100644 --- a/Migration2025.mdwn +++ b/Migration2025.mdwn @@ -1,4 +1,4 @@ -Scratchpad for planning ideas for the fall-2025 migration. +Scratchpad for planning ideas for the winter-2025 migration. # Hardware resources
diff --git a/Migration2025.mdwn b/Migration2025.mdwn index 9d2d178..4937779 100644 --- a/Migration2025.mdwn +++ b/Migration2025.mdwn @@ -4,9 +4,10 @@ Scratchpad for planning ideas for the fall-2025 migration. * New SFC server: server1.sourceware.org: Dell R740, 1.5TB RAM, 6*3.8TB SSD, 20+core * Old RH servers: server2.sourceware.org, server3.sourceware.org -* OSUOSL servers: 2 x86_64 smallish servers for container builders (48G/62G mem, 260GB/450GB disk, 16/24 cores) - snapshots server (8G mem, 620GB disk, 4 cores) - 2 arm64 builders (16G/32G mem, 200GB/320GB disk, 8/16 cores). +* OSUOSL servers: + * 2 x86_64 smallish servers for container builders (48G/62G mem, 260GB/450GB disk, 16/24 cores) + * snapshots server (8G mem, 620GB disk, 4 cores) + * 2 arm64 builders (16G/32G mem, 200GB/320GB disk, 8/16 cores). # Software
Add details for OSUOSL servers
diff --git a/Migration2025.mdwn b/Migration2025.mdwn index 48042bf..9d2d178 100644 --- a/Migration2025.mdwn +++ b/Migration2025.mdwn @@ -2,9 +2,11 @@ Scratchpad for planning ideas for the fall-2025 migration. # Hardware resources -* New SFC server: future server1.sourceware.org: Dell R740, 1.5TB RAM, 6*3.8TB SSD, 20+core +* New SFC server: server1.sourceware.org: Dell R740, 1.5TB RAM, 6*3.8TB SSD, 20+core * Old RH servers: server2.sourceware.org, server3.sourceware.org -* OSUOSL servers: 2 x86_64 smallish servers + etc. +* OSUOSL servers: 2 x86_64 smallish servers for container builders (48G/62G mem, 260GB/450GB disk, 16/24 cores) + snapshots server (8G mem, 620GB disk, 4 cores) + 2 arm64 builders (16G/32G mem, 200GB/320GB disk, 8/16 cores). # Software
Add current disk needs for bunsen
diff --git a/Migration2025.mdwn b/Migration2025.mdwn index 9e8d54e..48042bf 100644 --- a/Migration2025.mdwn +++ b/Migration2025.mdwn @@ -18,7 +18,8 @@ Guiding principles: no direct file sharing between/across VMs. * sourceware.org + gcc.gnu.org: a RHEL8? megaVM containing most current sourceware stuff; git repos, bugzilla; mysql; gitweb; postfix; rsync * if practical: mailman: a RHEL8 VM running mailman2 + public-inbox; postfix; mailing-list archives by git-over-inbox/http only -* patchwork + * We can upgrade this to something newer if we run mailmain2 in a container +* patchwork: * buildbot: (pushing bunsen data to sourceware:/git/bunsendb.git via cross-vm credential) * forge: maybe repatriate this from RH OSPO hosting service in entirety * forge-stage: a test environment for the above @@ -26,6 +27,7 @@ Guiding principles: no direct file sharing between/across VMs. #### additional possible VMs: * bunsen: cloning bunsen data from sourceware:/git/bunsendb.git; running indexer & web-ui + Disk need ~140GB+ for bunsendb git and ~300GB+ for bunsenweb and ~45GB+ for bunsenql (~485GB+ total data). * gitweb/cgit; cloning git repos from sourceware:/git/ * moinmoin: wikis * cygwin?
diff --git a/Migration2025.mdwn b/Migration2025.mdwn index ddbb80a..9e8d54e 100644 --- a/Migration2025.mdwn +++ b/Migration2025.mdwn @@ -9,23 +9,26 @@ Scratchpad for planning ideas for the fall-2025 migration. # Software We wish to split the unitary sourceware.org server into multiple VMs, as practical. +This will likely mean that the bare metal servers '''will not be accessible''' to the public, only the VMs they host. We wish to maintain appearance of current operation model (maintainers can direct ssh to git-push, URLs preserved, etc.). ## VMs on server1 Guiding principles: no direct file sharing between/across VMs. -* sourceware: a RHEL8? megaVM containing most current sourceware stuff; git repos, bugzilla; mysql; gitweb; postfix; rsync -* mailman: a RHEL8 VM running mailman2 + public-inbox; postfix; mailing-list archives by git-over-inbox/http only +* sourceware.org + gcc.gnu.org: a RHEL8? megaVM containing most current sourceware stuff; git repos, bugzilla; mysql; gitweb; postfix; rsync +* if practical: mailman: a RHEL8 VM running mailman2 + public-inbox; postfix; mailing-list archives by git-over-inbox/http only * patchwork * buildbot: (pushing bunsen data to sourceware:/git/bunsendb.git via cross-vm credential) * forge: maybe repatriate this from RH OSPO hosting service in entirety +* forge-stage: a test environment for the above -#### possible VMs: +#### additional possible VMs: * bunsen: cloning bunsen data from sourceware:/git/bunsendb.git; running indexer & web-ui * gitweb/cgit; cloning git repos from sourceware:/git/ * moinmoin: wikis +* cygwin? ## Services on server1 host @@ -34,14 +37,6 @@ Guiding principles: no direct file sharing between/across VMs. * vm hosting: as above * vm backup: LVM snapshots of VM virtual disks * rsyslog or journal-remote: log collection from all the VMs; ideally even httpd logs -* outgoing postfix: for outgoing email, smarthost for VMs; DKIM/ARC signatures -* incoming postfix: spamassassin; routing incoming mailing list email to mailman VM, others to sourceware VM; maybe spamassassin mysql -* admin-ssh: on a non-default port, to allow rootish logons to host -* ssh: port-forwarding to sourceware VM ssh -* httpd: reverse-proxying to the various VMs; https termination (incl. letsencrypt cert generation); anubis -* dns/dhcpd?, including FOO.vm.sourceware.org for local access to VMs -* port-forward incoming tcp ports (9418 to sourceware vm, 99* to buildbot, etc.) -* fail2ban; ***but*** ssh/etc. failures in port-forwarded VMs can't easily be mapped back to host, even with shared logfiles ## Services on server2 + server3
diff --git a/Migration2025.mdwn b/Migration2025.mdwn index f460136..ddbb80a 100644 --- a/Migration2025.mdwn +++ b/Migration2025.mdwn @@ -24,7 +24,7 @@ Guiding principles: no direct file sharing between/across VMs. #### possible VMs: * bunsen: cloning bunsen data from sourceware:/git/bunsendb.git; running indexer & web-ui -* gitweb/cgit; cloning git repos from sourceware:/git/bunsendb.git +* gitweb/cgit; cloning git repos from sourceware:/git/ * moinmoin: wikis ## Services on server1 host
diff --git a/Migration2025.mdwn b/Migration2025.mdwn index d468b68..f460136 100644 --- a/Migration2025.mdwn +++ b/Migration2025.mdwn @@ -33,7 +33,7 @@ Guiding principles: no direct file sharing between/across VMs. * no developer/project/maintainer logins at all * vm hosting: as above * vm backup: LVM snapshots of VM virtual disks -* rsyslog: log collection from all the VMs +* rsyslog or journal-remote: log collection from all the VMs; ideally even httpd logs * outgoing postfix: for outgoing email, smarthost for VMs; DKIM/ARC signatures * incoming postfix: spamassassin; routing incoming mailing list email to mailman VM, others to sourceware VM; maybe spamassassin mysql * admin-ssh: on a non-default port, to allow rootish logons to host
diff --git a/Migration2025.mdwn b/Migration2025.mdwn index 1106e53..d468b68 100644 --- a/Migration2025.mdwn +++ b/Migration2025.mdwn @@ -41,7 +41,7 @@ Guiding principles: no direct file sharing between/across VMs. * httpd: reverse-proxying to the various VMs; https termination (incl. letsencrypt cert generation); anubis * dns/dhcpd?, including FOO.vm.sourceware.org for local access to VMs * port-forward incoming tcp ports (9418 to sourceware vm, 99* to buildbot, etc.) -* fail2ban +* fail2ban; ***but*** ssh/etc. failures in port-forwarded VMs can't easily be mapped back to host, even with shared logfiles ## Services on server2 + server3
diff --git a/Migration2025.mdwn b/Migration2025.mdwn index dee2bc9..1106e53 100644 --- a/Migration2025.mdwn +++ b/Migration2025.mdwn @@ -41,6 +41,7 @@ Guiding principles: no direct file sharing between/across VMs. * httpd: reverse-proxying to the various VMs; https termination (incl. letsencrypt cert generation); anubis * dns/dhcpd?, including FOO.vm.sourceware.org for local access to VMs * port-forward incoming tcp ports (9418 to sourceware vm, 99* to buildbot, etc.) +* fail2ban ## Services on server2 + server3
diff --git a/Migration2025.mdwn b/Migration2025.mdwn index 174a17f..dee2bc9 100644 --- a/Migration2025.mdwn +++ b/Migration2025.mdwn @@ -33,12 +33,13 @@ Guiding principles: no direct file sharing between/across VMs. * no developer/project/maintainer logins at all * vm hosting: as above * vm backup: LVM snapshots of VM virtual disks +* rsyslog: log collection from all the VMs * outgoing postfix: for outgoing email, smarthost for VMs; DKIM/ARC signatures * incoming postfix: spamassassin; routing incoming mailing list email to mailman VM, others to sourceware VM; maybe spamassassin mysql * admin-ssh: on a non-default port, to allow rootish logons to host * ssh: port-forwarding to sourceware VM ssh * httpd: reverse-proxying to the various VMs; https termination (incl. letsencrypt cert generation); anubis -* dns +* dns/dhcpd?, including FOO.vm.sourceware.org for local access to VMs * port-forward incoming tcp ports (9418 to sourceware vm, 99* to buildbot, etc.) ## Services on server2 + server3
diff --git a/Migration2025.mdwn b/Migration2025.mdwn index 6b081df..174a17f 100644 --- a/Migration2025.mdwn +++ b/Migration2025.mdwn @@ -15,12 +15,18 @@ We wish to maintain appearance of current operation model (maintainers can direc Guiding principles: no direct file sharing between/across VMs. -* sourceware: a RHEL8? megaVM containing most current sourceware stuff; git repos, bugzilla; mysql; gitweb; postfix -* mailman: a RHEL8 VM running mailman2 + public-inbox; postfix -* patchwork: -* buildbot + bunsen: (pushing bunsen data to sourceware:/git/bunsendb.git via cross-vm credential) +* sourceware: a RHEL8? megaVM containing most current sourceware stuff; git repos, bugzilla; mysql; gitweb; postfix; rsync +* mailman: a RHEL8 VM running mailman2 + public-inbox; postfix; mailing-list archives by git-over-inbox/http only +* patchwork +* buildbot: (pushing bunsen data to sourceware:/git/bunsendb.git via cross-vm credential) * forge: maybe repatriate this from RH OSPO hosting service in entirety +#### possible VMs: + +* bunsen: cloning bunsen data from sourceware:/git/bunsendb.git; running indexer & web-ui +* gitweb/cgit; cloning git repos from sourceware:/git/bunsendb.git +* moinmoin: wikis + ## Services on server1 host * rhel10
diff --git a/Migration2025.mdwn b/Migration2025.mdwn new file mode 100644 index 0000000..6b081df --- /dev/null +++ b/Migration2025.mdwn @@ -0,0 +1,41 @@ +Scratchpad for planning ideas for the fall-2025 migration. + +# Hardware resources + +* New SFC server: future server1.sourceware.org: Dell R740, 1.5TB RAM, 6*3.8TB SSD, 20+core +* Old RH servers: server2.sourceware.org, server3.sourceware.org +* OSUOSL servers: 2 x86_64 smallish servers + etc. + +# Software + +We wish to split the unitary sourceware.org server into multiple VMs, as practical. +We wish to maintain appearance of current operation model (maintainers can direct ssh to git-push, URLs preserved, etc.). + +## VMs on server1 + +Guiding principles: no direct file sharing between/across VMs. + +* sourceware: a RHEL8? megaVM containing most current sourceware stuff; git repos, bugzilla; mysql; gitweb; postfix +* mailman: a RHEL8 VM running mailman2 + public-inbox; postfix +* patchwork: +* buildbot + bunsen: (pushing bunsen data to sourceware:/git/bunsendb.git via cross-vm credential) +* forge: maybe repatriate this from RH OSPO hosting service in entirety + +## Services on server1 host + +* rhel10 +* no developer/project/maintainer logins at all +* vm hosting: as above +* vm backup: LVM snapshots of VM virtual disks +* outgoing postfix: for outgoing email, smarthost for VMs; DKIM/ARC signatures +* incoming postfix: spamassassin; routing incoming mailing list email to mailman VM, others to sourceware VM; maybe spamassassin mysql +* admin-ssh: on a non-default port, to allow rootish logons to host +* ssh: port-forwarding to sourceware VM ssh +* httpd: reverse-proxying to the various VMs; https termination (incl. letsencrypt cert generation); anubis +* dns +* port-forward incoming tcp ports (9418 to sourceware vm, 99* to buildbot, etc.) + +## Services on server2 + server3 + +* ? +* LVM snapshot copies from server1
diff --git a/index.mdwn b/index.mdwn
index ae5c2ac..305176f 100644
--- a/index.mdwn
+++ b/index.mdwn
@@ -55,7 +55,11 @@
* gerrit
* recipes:
* creating new [[user|OSNewUser]], [[project|OSNewProject]], [[git repo|OSNewGitRepo]], [[mailing list|OSNewMailingList]], [[web hosting site|OSNewWebHosting]], [[moinmoin wiki|OSNewMoinWiki]], bugzilla, [[patchwork]]
+
+#### Migration
+
* [[2019 migration status|MigrationStatus]] [[2019 migration details|MigrationWorkItems]]
+* [[2025 migration plans|Migration2025]]
<font size="-1">Archive of old documentation, from [[y2000 era|OldInfo2000]], from [[y2010 era|OldInfo2010]].</font>
diff --git a/index.mdwn b/index.mdwn
index b34dea6..ae5c2ac 100644
--- a/index.mdwn
+++ b/index.mdwn
@@ -32,7 +32,7 @@
* [[aging]]
* [request queue](https://sourceware.org/cgi-bin/pdw/queue.cgi)
* approval policies:
- * gcc: [maintainers](https://gcc.gnu.org/gitwrite.html), approvers: maintainers, not just write-after-approval
+ * gcc: [maintainers](https://gcc.gnu.org/gitwrite.html), approvers: maintainers+reviewers, not just write-after-approval
* glibc: [becoming a maintainer](https://sourceware.org/glibc/wiki/MAINTAINERS#Becoming_a_maintainer_.28developer.29) - approvers: stewards
* problem solving
* failures [[disk|OSPSDiskFailure]]
diff --git a/patchwork.mdwn b/patchwork.mdwn new file mode 100644 index 0000000..e629101 --- /dev/null +++ b/patchwork.mdwn @@ -0,0 +1,11 @@ +To add a new project to https://patchwork.sourceware.org/ you need someone with Django admin access. +https://patchwork.readthedocs.io/en/latest/usage/overview/ + +To add a mailinglist subscription you need someone with a patchwork admin account access. + +The mailinglists are handled through the .forward file of the patchwork admin account. +https://patchwork.readthedocs.io/en/latest/deployment/installation/#mail-transfer-agent-mta + +* Add yourself to the patchwork admin account .forward file +* Sign up for the mailing list and confirm via the patchwork email. +* Remove yourself from the .forward
diff --git a/index.mdwn b/index.mdwn
index 91b936b..b34dea6 100644
--- a/index.mdwn
+++ b/index.mdwn
@@ -50,11 +50,11 @@
* bugzilla
* [[moinmoinwiki]]
* ikiwiki
- * patchworks
+ * [[patchwork]]
* mysql
* gerrit
* recipes:
- * creating new [[user|OSNewUser]], [[project|OSNewProject]], [[git repo|OSNewGitRepo]], [[mailing list|OSNewMailingList]], [[web hosting site|OSNewWebHosting]], [[moinmoin wiki|OSNewMoinWiki]], bugzilla
+ * creating new [[user|OSNewUser]], [[project|OSNewProject]], [[git repo|OSNewGitRepo]], [[mailing list|OSNewMailingList]], [[web hosting site|OSNewWebHosting]], [[moinmoin wiki|OSNewMoinWiki]], bugzilla, [[patchwork]]
* [[2019 migration status|MigrationStatus]] [[2019 migration details|MigrationWorkItems]]
<font size="-1">Archive of old documentation, from [[y2000 era|OldInfo2000]], from [[y2010 era|OldInfo2010]].</font>
Created forge group, mailinglist and git repo
diff --git a/SystemStatusHistory.mdwn b/SystemStatusHistory.mdwn index 52cb947..dbd7b2b 100644 --- a/SystemStatusHistory.mdwn +++ b/SystemStatusHistory.mdwn @@ -1,5 +1,6 @@ # service news +* 2024-10-20 - Created forge group, mailinglist and git repo. * 2024-08-31 - DoS on git:// protocol took anonymous-git down for a few hours; https: & ssh: continued working. * 2024-06-03 - Level3 internet backbone DNS PTR screwup in Europe made some outgoing sourceware+subnet mail undeliverable in Europe. * 2023-12-04 - Fiber cut & ISP routing problems resulting in partial network unreachability.
Add instructions for spam removal of inbox
diff --git a/Email.mdwn b/Email.mdwn
index d9779fc..becde03 100644
--- a/Email.mdwn
+++ b/Email.mdwn
@@ -26,6 +26,11 @@
# chmod 640 FILE.html # not 000, leads to /var/log/mailman/error
</pre>
+* To remove an inbox.sourceware.org message from the html,nntp,imap archives as inbox admin do:
+<pre>
+ $ curl https://inbox.sourceware.org/..../raw | public-inbox-learn spam
+<pre>
+
### pre-2019 historical information
diff --git a/SystemStatusHistory.mdwn b/SystemStatusHistory.mdwn index 5b22f69..52cb947 100644 --- a/SystemStatusHistory.mdwn +++ b/SystemStatusHistory.mdwn @@ -1,5 +1,7 @@ # service news +* 2024-08-31 - DoS on git:// protocol took anonymous-git down for a few hours; https: & ssh: continued working. +* 2024-06-03 - Level3 internet backbone DNS PTR screwup in Europe made some outgoing sourceware+subnet mail undeliverable in Europe. * 2023-12-04 - Fiber cut & ISP routing problems resulting in partial network unreachability. * 2023-11-20 - Fiber cut & ISP routing problems resulting in partial network unreachability. * 2023-10-15 - ARC milter configured.
diff --git a/DoS.mdwn b/DoS.mdwn index 4fe1613..47d61c1 100644 --- a/DoS.mdwn +++ b/DoS.mdwn @@ -9,7 +9,7 @@ fail2ban-client status postfix fail2ban-client status moinmoin </code></pre> -* httpd blocklist (no longer used) +* httpd blocklist <pre><code> /etc/httpd/conf.d/block.include*
diff --git a/UseridPolicy.mdwn b/UseridPolicy.mdwn
index 9dd3fcc..ee5a521 100644
--- a/UseridPolicy.mdwn
+++ b/UseridPolicy.mdwn
@@ -29,8 +29,7 @@ These get normal userids (>= 1000), and one of the following gids to identify th
`# usermod -g 1001 -G "" -s /sbin/nologin USERID`
-No password (`useradd -p x`), no `/etc/shadow` record either: ssh-only logons for now.
-
+Our accounts use no password (`useradd -p x`), no `/etc/shadow` record either: ssh-only logons.
`# getent passwd USERID; id USERID`
diff --git a/UseridPolicy.mdwn b/UseridPolicy.mdwn
index 6505e25..9dd3fcc 100644
--- a/UseridPolicy.mdwn
+++ b/UseridPolicy.mdwn
@@ -18,10 +18,15 @@ These get normal userids (>= 1000), and one of the following gids to identify th
* 1000 (developer, the usual)
* 1001 (absent or retired developers, reserving the userid)
+
`# usermod -g 1001 USERID`
+
This can be reversed with
+
`# usermod -g 1000 USERID`
+
For users who are likely gone forever, consider `/sbin/nologin` as the shell.
+
`# usermod -g 1001 -G "" -s /sbin/nologin USERID`
No password (`useradd -p x`), no `/etc/shadow` record either: ssh-only logons for now.
diff --git a/UseridPolicy.mdwn b/UseridPolicy.mdwn index 007da27..6505e25 100644 --- a/UseridPolicy.mdwn +++ b/UseridPolicy.mdwn @@ -17,14 +17,16 @@ Since these are maintained by sourceware users and/or overseers, these get norma These get normal userids (>= 1000), and one of the following gids to identify their general role: * 1000 (developer, the usual) -* 1001 (emeritus developer whom we're not expecting to ever log on again, just reserving the name;) - Such users should have `/sbin/nologin` as the shell. - - # usermod -g 1001 -G '' -s /sbin/nologin USERID +* 1001 (absent or retired developers, reserving the userid) + `# usermod -g 1001 USERID` + This can be reversed with + `# usermod -g 1000 USERID` + For users who are likely gone forever, consider `/sbin/nologin` as the shell. + `# usermod -g 1001 -G "" -s /sbin/nologin USERID` No password (`useradd -p x`), no `/etc/shadow` record either: ssh-only logons for now. - # getent passwd USERID; id USERID + `# getent passwd USERID; id USERID` # group assignments
diff --git a/aging.mdwn b/aging.mdwn new file mode 100644 index 0000000..8778063 --- /dev/null +++ b/aging.mdwn @@ -0,0 +1,9 @@ +Account aging! + +* /sourceware/infra/bin/list-ssh-login produces a report of every gid=1000 user who has or hasn't logged on, as per given /var/log/secure* log files (containing ssh authentication records) + +* /sourceware/infra/bin/list-ssh-login-email mass-emails those who are passive. + +* /sourceware/infra/bin/user-retire retires a single given user + +NB: the above scripts deal with access at the individual account level, not the shared-account per-key level.
diff --git a/index.mdwn b/index.mdwn
index 538a9bd..91b936b 100644
--- a/index.mdwn
+++ b/index.mdwn
@@ -29,6 +29,7 @@
* server3.sourceware.org - warm backup
* project hosting policies
* account policies
+ * [[aging]]
* [request queue](https://sourceware.org/cgi-bin/pdw/queue.cgi)
* approval policies:
* gcc: [maintainers](https://gcc.gnu.org/gitwrite.html), approvers: maintainers, not just write-after-approval