Postal v1.1: crash resume, A/B subjects, send-hours window, segments, suppression, unsub endpoint, watchdog, Proton SMTP
This commit is contained in:
Hermes
68
README.md
Normal file
68
README.md
Normal file
@@ -0,0 +1,68 @@
|
|||||||
|
# Postal
|
||||||
|
|
||||||
|
Ollama-personalized email campaign sender for drjones' homelab. Windows-native (VPS), SQLite-backed,
|
||||||
|
LLM personalization via gemma4 on nightmare over Tailscale, delivery via SMTP (Proton) or Amazon SES.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
[VPS 64.224.17.129: C:\Mailer] [nightmare 100.103.34.0]
|
||||||
|
Flask GUI :8899 (localhost) <--TS--> ollama-shim :11440 (non-chunked proxy)
|
||||||
|
engine.py (send loop, SQLite) └─> Ollama :11434 mgraffam/gemma4-heretic:12b
|
||||||
|
delivery: SMTP (Proton) or SES (boto3)
|
||||||
|
```
|
||||||
|
|
||||||
|
- **ollama-shim** exists because Ollama's chunked streaming stalls over this tailnet path;
|
||||||
|
the shim returns one plain body. systemd unit `ollama-shim.service` on nightmare.
|
||||||
|
- All state in `mailer.db` (subscribers, sends, campaigns, ab_variants, suppression).
|
||||||
|
|
||||||
|
## Features
|
||||||
|
|
||||||
|
- CSV import: `email,firstname,interest` per line
|
||||||
|
- Per-subscriber intro written by gemma4 (thinking-model aware: extracts quoted greeting from CoT)
|
||||||
|
- A/B subject variants — gemma4 rewrites the subject 2×; rotated evenly; per-variant sent counts
|
||||||
|
- Send-hours window (`send_hours: 9-18` server-local) — sleeps outside the window
|
||||||
|
- Segment targeting (`segment_interest` LIKE match)
|
||||||
|
- Suppression list (imports + public `/unsub/<email>` endpoint)
|
||||||
|
- Rate limiting (emails/min), batch pauses, STOP switch
|
||||||
|
- Crash resume: queued sends persist in SQLite; auto-resume on boot/startup
|
||||||
|
- Watchdog task (`PostalWatchdog`, every 5 min) restarts the app if :8899 stops answering
|
||||||
|
- Test-send button (personalizes + sends one real email)
|
||||||
|
- Tunables UI with hover explanations: model, rate, batch, hours, send mode, SMTP creds, prompt
|
||||||
|
|
||||||
|
## Setup (Windows VPS)
|
||||||
|
|
||||||
|
```
|
||||||
|
cd C:\Mailer
|
||||||
|
python -m venv --system-site-packages venv
|
||||||
|
venv\Scripts\pip install flask requests boto3
|
||||||
|
schtasks /create /tn MailerApp /tr "C:\Mailer\start.bat" /sc onstart /ru SYSTEM /rl HIGHEST /f
|
||||||
|
schtasks /create /tn PostalWatchdog /tr "C:\Mailer\watchdog.bat" /sc minute /mo 5 /ru SYSTEM /rl HIGHEST /f
|
||||||
|
```
|
||||||
|
|
||||||
|
## Config (Tunables in the UI)
|
||||||
|
|
||||||
|
| key | default | note |
|
||||||
|
|---|---|---|
|
||||||
|
| ollama_url | http://100.103.34.0:11440 | nightmare via shim |
|
||||||
|
| ollama_model | mgraffam/gemma4-heretic:12b | thinking model, handled |
|
||||||
|
| send_hours | 9-18 | server-local window |
|
||||||
|
| rate_per_minute | 60 | keep ≤60 while warming reputation |
|
||||||
|
| send_mode | smtp | `smtp` (Proton, ~2000/day paid) or `ses` (needs AWS IAM key + verified domain) |
|
||||||
|
| smtp_host/user/pass | proton | pass = Proton SMTP token (Settings → SMTP tokens), paid plan required |
|
||||||
|
|
||||||
|
## Scaling to 100K+
|
||||||
|
|
||||||
|
- Use SES (~$1/10K emails), verify the sending domain (SPF/DKIM via Cloudflare), start in sandbox,
|
||||||
|
request production access, then warm up: 50/day → double daily → cap ~10K/day before the big push.
|
||||||
|
- Bounce/complaint webhook handling is the next build item; until then monitor SES dashboard daily.
|
||||||
|
|
||||||
|
## Files
|
||||||
|
|
||||||
|
| file | purpose |
|
||||||
|
|---|---|
|
||||||
|
| app.py | Flask GUI + API |
|
||||||
|
| engine.py | send loop, Ollama client, Sender (smtp/ses), SQLite schema |
|
||||||
|
| start.bat | launcher (opens browser, runs app) |
|
||||||
|
| watchdog.bat | health-check + auto-restart |
|
||||||
|
| ollama_shim.py | deploys to nightmare — non-chunked Ollama proxy on tailnet |
|
||||||
361
app.py
Normal file
361
app.py
Normal file
@@ -0,0 +1,361 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Mailer GUI — Flask web app, localhost:8899 on the VPS (RDP into it and open the browser)."""
|
||||||
|
import os, sys, csv, io, json
|
||||||
|
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
from flask import Flask, request, jsonify, render_template_string, redirect, url_for
|
||||||
|
from engine import (db, init_db, load_config, save_config, get_engine, render_subject)
|
||||||
|
|
||||||
|
APP_DIR = os.path.dirname(os.path.abspath(__file__))
|
||||||
|
app = Flask(__name__)
|
||||||
|
init_db()
|
||||||
|
|
||||||
|
PAGE = r"""<!doctype html><html><head><meta charset=utf-8><title>Mailer</title>
|
||||||
|
<style>
|
||||||
|
:root{--bg:#0d0f1a;--panel:#151827;--panel2:#1c2033;--line:#2a2f4a;--txt:#e8eaf6;--dim:#9298b8;
|
||||||
|
--acc:#7c6cf5;--acc2:#5e8bff;--ok:#4ade80;--bad:#f87171;--warn:#fbbf24}
|
||||||
|
*{margin:0;padding:0;box-sizing:border-box}
|
||||||
|
body{background:var(--bg);color:var(--txt);font:15px/1.5 'Segoe UI',system-ui,sans-serif;padding:20px}
|
||||||
|
.wrap{max-width:1180px;margin:0 auto}
|
||||||
|
h1{font-size:26px;margin-bottom:4px}h1 .dot{display:inline-block;width:10px;height:10px;border-radius:50%;background:var(--ok);margin-left:8px;animation:pulse 2s infinite}
|
||||||
|
@keyframes pulse{50%{opacity:.3}}
|
||||||
|
.sub{color:var(--dim);margin-bottom:18px}
|
||||||
|
.grid{display:grid;grid-template-columns:1fr 1fr;gap:14px}
|
||||||
|
.card{background:var(--panel);border:1px solid var(--line);border-radius:14px;padding:16px;margin-bottom:14px}
|
||||||
|
.card h2{font-size:15px;color:var(--acc);margin-bottom:10px;text-transform:uppercase;letter-spacing:.6px}
|
||||||
|
.full{grid-column:1/-1}
|
||||||
|
label{display:block;font-size:12.5px;color:var(--dim);margin:8px 0 3px}
|
||||||
|
input[type=text],input[type=number],textarea,select{width:100%;background:var(--panel2);border:1px solid var(--line);border-radius:8px;color:var(--txt);padding:8px 10px;font-size:14px}
|
||||||
|
textarea{min-height:110px;resize:vertical;font-family:Consolas,monospace;font-size:13px}
|
||||||
|
button{background:linear-gradient(135deg,var(--acc),var(--acc2));border:0;border-radius:9px;color:#fff;padding:9px 20px;font-size:14.5px;font-weight:600;cursor:pointer;margin-top:10px}
|
||||||
|
button:hover{filter:brightness(1.15)}
|
||||||
|
button.stop{background:linear-gradient(135deg,#e05555,#c22f2f)}
|
||||||
|
button.small{padding:5px 12px;font-size:12.5px;margin-top:6px}
|
||||||
|
.row{display:flex;gap:10px}.row>*{flex:1}
|
||||||
|
table{width:100%;border-collapse:collapse;font-size:13.5px}
|
||||||
|
th{color:var(--dim);text-align:left;font-weight:600;padding:6px 8px;border-bottom:1px solid var(--line)}
|
||||||
|
td{padding:6px 8px;border-bottom:1px solid var(--line)}
|
||||||
|
.stats{display:flex;gap:12px;flex-wrap:wrap;margin:10px 0}
|
||||||
|
.stat{background:var(--panel2);border-radius:11px;padding:10px 18px;text-align:center;min-width:110px}
|
||||||
|
.stat .n{font-size:26px;font-weight:700}
|
||||||
|
.stat .l{font-size:11.5px;color:var(--dim);text-transform:uppercase;letter-spacing:.5px}
|
||||||
|
.badge{display:inline-block;padding:2px 9px;border-radius:20px;font-size:11.5px;font-weight:600}
|
||||||
|
.b-sent{background:#14532d;color:var(--ok)} .b-queued{background:#1e3a8a;color:#93c5fd}
|
||||||
|
.b-failed{background:#7f1d1d;color:var(--bad)} .b-personalizing{background:#78350f;color:var(--warn)}
|
||||||
|
.tip{cursor:help;border-bottom:1px dotted var(--dim)}
|
||||||
|
#log{background:#0a0c14;border:1px solid var(--line);border-radius:8px;padding:10px;font:12px Consolas,monospace;max-height:180px;overflow-y:auto;color:#8f9}
|
||||||
|
.bar{height:8px;background:var(--panel2);border-radius:6px;overflow:hidden;margin:8px 0}
|
||||||
|
.bar>div{height:100%;background:linear-gradient(90deg,var(--acc),var(--acc2));transition:width .6s}
|
||||||
|
.tun{display:flex;align-items:center;gap:8px;margin:6px 0}
|
||||||
|
.tun input[type=number]{width:90px}
|
||||||
|
</style></head><body><div class=wrap>
|
||||||
|
<h1>Postal<span class=dot id=livedot></span></h1>
|
||||||
|
<div class=sub>Ollama-personalized campaign mailer — subscribers only, SES delivery</div>
|
||||||
|
|
||||||
|
<div class=stats id=stats></div>
|
||||||
|
<div class=bar><div id=prog style="width:0%"></div></div>
|
||||||
|
|
||||||
|
<div class=grid>
|
||||||
|
<div class="card full">
|
||||||
|
<h2>Engine</h2>
|
||||||
|
<div id=engineline>Loading…</div>
|
||||||
|
<button onclick=start() id=btnstart>Start campaign</button>
|
||||||
|
<button class=stop onclick=stopEngine()>STOP</button>
|
||||||
|
<span id=stopnote style="color:var(--warn);margin-left:10px;display:none">Stop requested — finishing current email…</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class=card>
|
||||||
|
<h2><span class=tip title="Plain-text or CSV: one email per line, or email,firstname,lastname,interest">Subscribers</span></h2>
|
||||||
|
<textarea id=subs placeholder="email,firstname,interest jane@example.com,Jane,bitcoin mining bob@example.com,Bob,"></textarea>
|
||||||
|
<button class=small onclick=importSubs()>Import</button> <span id=impnote></span>
|
||||||
|
<div style=margin-top:8px id=subcount></div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class=card>
|
||||||
|
<h2>Campaign</h2>
|
||||||
|
<label>Subject</label>
|
||||||
|
<input type=text id=subject placeholder="Hey {{first_name}} — quick one">
|
||||||
|
<label>Segment: only subscribers whose interest contains… (blank = everyone)</label>
|
||||||
|
<input type=text id=segment placeholder="bitcoin">
|
||||||
|
<label>Body (HTML). Tags: {{first_name}} {{interest}} {{from_name}} — and <b>{{intro}}</b> = the AI-written personal bit</label>
|
||||||
|
<textarea id=body placeholder="<p>{{intro}}</p> <p>Check out what we shipped: …</p>"></textarea>
|
||||||
|
<button class=small onclick=saveCampaign()>Save campaign</button> <span id=campnote></span>
|
||||||
|
<div class=row style=margin-top:8px>
|
||||||
|
<input type=text id=testemail placeholder="your@email — test send to">
|
||||||
|
<button class=small onclick=testSend()>Test send</button>
|
||||||
|
</div><div id=testnote style="color:var(--warn);font-size:12.5px"></div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class=card>
|
||||||
|
<h2><span class=tip title="Settings for the sending engine. Hover anything underlined.">Tunables</span></h2>
|
||||||
|
<div class=tun><label style=margin:0>Ollama URL</label><input type=text id=c_ollama_url>
|
||||||
|
<span class=tip title="Where the LLM brain lives. Tailscale IP of nightmare (100.x). The app checks it live.">?</span></div>
|
||||||
|
<div class=tun><label style=margin:0>Model</label><input type=text id=c_ollama_model>
|
||||||
|
<span class=tip title="Ollama model name, e.g. gemma4:latest. Run 'ollama list' on nightmare to see options.">?</span></div>
|
||||||
|
<div class=tun><label style=margin:0>Rate/min</label><input type=number id=c_rate_per_minute>
|
||||||
|
<span class=tip title="Emails per minute. 60/min = 3600/hr. Keep at or under 60 while warming your sender reputation.">?</span></div>
|
||||||
|
<div class=tun><label style=margin:0>Batch size</label><input type=number id=c_batch_size>
|
||||||
|
<span class=tip title="Emails between cooldown pauses.">?</span></div>
|
||||||
|
<div class=tun><label style=margin:0>Pause secs</label><input type=number id=c_pause_between_batches_sec>
|
||||||
|
<span class=tip title="Cooldown between batches — lets SES API breathe and keeps deliverability high.">?</span></div>
|
||||||
|
<div class=tun><label><input type=checkbox id=c_personalize> Personalize with Ollama</label>
|
||||||
|
<span class=tip title="Off = sends the template as-is. On = writes a custom intro per person.">?</span></div>
|
||||||
|
<div class=tun><label style=margin:0>From name</label><input type=text id=c_from_name></div>
|
||||||
|
<div class=tun><label style=margin:0>From email (SES-verified)</label><input type=text id=c_from_email>
|
||||||
|
<span class=tip title="Must be a domain/email verified in Amazon SES first.">?</span></div>
|
||||||
|
<div class=tun><label style=margin:0>Reply-to</label><input type=text id=c_reply_to></div>
|
||||||
|
<div class=tun><label style=margin:0>AWS region</label><input type=text id=c_ses_region></div>
|
||||||
|
<div class=tun><label style=margin:0>Send mode</label><select id=c_send_mode><option value=smtp>SMTP (works now)</option><option value=ses>SES API (needs AWS keys)</option></select>
|
||||||
|
<span class=tip title="SMTP = your iCloud account, fine for tests/small sends (iCloud caps ~200/day). SES = Amazon, built for 100K+.">?</span></div>
|
||||||
|
<div class=tun><label style=margin:0>SMTP host</label><input type=text id=c_smtp_host></div>
|
||||||
|
<div class=tun><label style=margin:0>SMTP port</label><input type=number id=c_smtp_port></div>
|
||||||
|
<div class=tun><label style=margin:0>SMTP user</label><input type=text id=c_smtp_user></div>
|
||||||
|
<div class=tun><label style=margin:0>SMTP app password</label><input type=text id=c_smtp_pass></div>
|
||||||
|
<label>Personalization prompt</label>
|
||||||
|
<textarea id=c_personalize_prompt style=min-height:70px></textarea>
|
||||||
|
<button class=small onclick=saveCfg()>Save settings</button> <span id=cfgnote></span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class=card>
|
||||||
|
<h2><span class=tip title="Emails that must NEVER be mailed: unsubscribes, complainers, past customers who opted out.">Suppression list</span></h2>
|
||||||
|
<textarea id=supp placeholder="one email per line"></textarea>
|
||||||
|
<button class=small onclick=doSuppress()>Add to suppression</button> <span id=suppnote></span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class=card>
|
||||||
|
<h2>Subject A/B (current campaign)</h2>
|
||||||
|
<div id=abdiv style="font-size:13px;color:var(--dim)">no data yet</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="card full">
|
||||||
|
<h2>Recent sends</h2>
|
||||||
|
<table id=recent><tr><th>email</th><th>status</th><th>subject</th><th>time</th></tr></table>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<script>
|
||||||
|
async function j(url, body){const o = body?{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify(body)}:{}; const r = await fetch(url,o); return r.json();}
|
||||||
|
async function refresh(){
|
||||||
|
const st = await j('/api/status');
|
||||||
|
const c = st.counts||{};
|
||||||
|
const sent=c.sent||0, queued=c.queued||0, failed=c.failed||0, pers=c.personalizing||0;
|
||||||
|
const total = st.total|| (sent+queued+failed+pers);
|
||||||
|
let h = `<div class=stat><div class=n>${total||'-'}</div><div class=l>queue</div></div>
|
||||||
|
<div class=stat><div class=n style=color:var(--ok)>${sent}</div><div class=l>sent</div></div>
|
||||||
|
<div class=stat><div class=n style=color:var(--warn)>${pers}</div><div class=l>writing…</div></div>
|
||||||
|
<div class=stat><div class=n>${queued}</div><div class=l>waiting</div></div>
|
||||||
|
<div class=stat><div class=n style=color:var(--bad)>${failed}</div><div class=l>failed</div></div>`;
|
||||||
|
document.getElementById('stats').innerHTML=h;
|
||||||
|
const pct = total? Math.round(100*(sent+failed)/total):0;
|
||||||
|
document.getElementById('prog').style.width=pct+'%';
|
||||||
|
document.getElementById('livedot').style.background = st.running?'var(--ok)':'var(--line)';
|
||||||
|
document.getElementById('engineline').innerHTML = st.running?
|
||||||
|
`Running — <b>${st.current}</b> (${pct}%)` : `Idle. ${st.stopNote?'<span style=color:var(--warn)>Stop pending.</span>':''}`;
|
||||||
|
const r2 = await j('/api/recent');
|
||||||
|
document.getElementById('recent').innerHTML = '<tr><th>email</th><th>status</th><th>subject</th><th>time</th></tr>' +
|
||||||
|
r2.map(s=>`<tr><td>${s.email}</td><td><span class="badge b-${s.status}">${s.status}</span></td><td>${(s.subject||'').slice(0,50)}</td><td>${s.sent_at||''}</td></tr>`).join('');
|
||||||
|
}
|
||||||
|
async function pullCfg(){
|
||||||
|
const c = await j('/api/config');
|
||||||
|
for (const k in c){ const el=document.getElementById('c_'+k); if(!el) continue;
|
||||||
|
if(el.type==='checkbox') el.checked=!!c[k]; else el.value=c[k]; }
|
||||||
|
}
|
||||||
|
async function saveCfg(){
|
||||||
|
const keys=['ollama_url','ollama_model','rate_per_minute','batch_size','pause_between_batches_sec','personalize','from_name','from_email','reply_to','ses_region','personalize_prompt','send_mode','smtp_host','smtp_port','smtp_user','smtp_pass','send_hours','ab_subjects'];
|
||||||
|
const out={};
|
||||||
|
keys.forEach(k=>{const el=document.getElementById('c_'+k); if(!el)return;
|
||||||
|
out[k]= el.type==='checkbox'? el.checked : (el.type==='number'? +el.value : el.value);});
|
||||||
|
await j('/api/config', out);
|
||||||
|
document.getElementById('cfgnote').textContent='saved ✓'; setTimeout(()=>document.getElementById('cfgnote').textContent='',2000);
|
||||||
|
}
|
||||||
|
async function importSubs(){
|
||||||
|
const text=document.getElementById('subs').value;
|
||||||
|
const r=await j('/api/import',{text});
|
||||||
|
document.getElementById('impnote').textContent=`${r.added} added, ${r.dupes} dupes`;
|
||||||
|
refreshCounts();
|
||||||
|
}
|
||||||
|
async function refreshCounts(){
|
||||||
|
const r=await j('/api/subcount'); document.getElementById('subcount').textContent=`${r.active} active subscribers in DB`;
|
||||||
|
}
|
||||||
|
async function saveCampaign(){
|
||||||
|
const r=await j('/api/campaign',{subject:document.getElementById('subject').value,body:document.getElementById('body').value,segment:document.getElementById('segment').value});
|
||||||
|
document.getElementById('campnote').textContent='saved ✓ (#'+r.id+')';
|
||||||
|
}
|
||||||
|
async function doSuppress(){
|
||||||
|
const text=document.getElementById('supp').value;
|
||||||
|
const r=await j('/api/suppress',{text});
|
||||||
|
document.getElementById('suppnote').textContent=`${r.added} added (${r.total} total suppressed)`;
|
||||||
|
document.getElementById('supp').value='';
|
||||||
|
}
|
||||||
|
async function refreshAB(){
|
||||||
|
const r=await j('/api/abstats');
|
||||||
|
const el=document.getElementById('abdiv');
|
||||||
|
if(!r.length){ el.textContent='no data yet'; return; }
|
||||||
|
el.innerHTML = r.map(v=>`<div>· ${v.subject} <b style=color:var(--txt)>${v.sent}</b> sent</div>`).join('');
|
||||||
|
}
|
||||||
|
async function testSend(){
|
||||||
|
const email=document.getElementById('testemail').value;
|
||||||
|
const note=document.getElementById('testnote');
|
||||||
|
note.textContent='sending (personalizing with gemma4 takes ~20s)...';
|
||||||
|
const r=await j('/api/testsend',{email});
|
||||||
|
note.textContent = r.ok? ('sent ✓ id='+r.id+' intro: '+r.intro) : ('FAILED: '+r.error);
|
||||||
|
if(r.ok) setTimeout(refresh, 1500);
|
||||||
|
}
|
||||||
|
async function start(){ const r=await j('/api/start',{}); if(!r.ok) alert(r.error); }
|
||||||
|
async function stopEngine(){ await j('/api/stop',{}); document.getElementById('stopnote').style.display='inline'; }
|
||||||
|
setInterval(refresh,3000); refresh(); pullCfg(); refreshCounts(); refreshAB(); setInterval(refreshAB,15000);
|
||||||
|
</script></body></html>"""
|
||||||
|
|
||||||
|
@app.route("/")
|
||||||
|
def home():
|
||||||
|
return render_template_string(PAGE)
|
||||||
|
|
||||||
|
@app.route("/api/status")
|
||||||
|
def api_status():
|
||||||
|
eng = get_engine()
|
||||||
|
st = eng.status()
|
||||||
|
st["stopNote"] = load_config().get("stopped", False)
|
||||||
|
return jsonify(st)
|
||||||
|
|
||||||
|
@app.route("/api/recent")
|
||||||
|
def api_recent():
|
||||||
|
c = db()
|
||||||
|
rows = c.execute("SELECT email,status,subject,sent_at FROM sends ORDER BY id DESC LIMIT 12").fetchall()
|
||||||
|
c.close()
|
||||||
|
return jsonify([dict(r) for r in rows])
|
||||||
|
|
||||||
|
@app.route("/api/config", methods=["GET", "POST"])
|
||||||
|
def api_config():
|
||||||
|
if request.method == "POST":
|
||||||
|
cfg = load_config()
|
||||||
|
incoming = request.json or {}
|
||||||
|
for k, v in incoming.items():
|
||||||
|
if k in cfg:
|
||||||
|
cfg[k] = v
|
||||||
|
# engine stop flag
|
||||||
|
cfg["stopped"] = bool(incoming.get("_stop", cfg.get("stopped")))
|
||||||
|
save_config(cfg)
|
||||||
|
return jsonify(ok=True)
|
||||||
|
return jsonify(load_config())
|
||||||
|
|
||||||
|
@app.route("/api/import", methods=["POST"])
|
||||||
|
def api_import():
|
||||||
|
text = (request.json or {}).get("text", "")
|
||||||
|
c = db()
|
||||||
|
added = dupes = 0
|
||||||
|
for raw in io.StringIO(text).readlines():
|
||||||
|
raw = raw.strip()
|
||||||
|
if not raw or raw.startswith("#"): continue
|
||||||
|
parts = [p.strip() for p in raw.replace(";", ",").split(",")]
|
||||||
|
email = parts[0].lower()
|
||||||
|
if "@" not in email: continue
|
||||||
|
fn = parts[1] if len(parts) > 1 else ""
|
||||||
|
ln = parts[2] if len(parts) > 2 else ""
|
||||||
|
interest = parts[3] if len(parts) > 3 else (parts[1] if len(parts) > 1 and "@" not in parts[1] and len(parts) <= 2 else "")
|
||||||
|
try:
|
||||||
|
c.execute("INSERT INTO subscribers (email, first_name, last_name, interest) VALUES (?,?,?,?)",
|
||||||
|
(email, fn, ln, interest))
|
||||||
|
added += 1
|
||||||
|
except Exception:
|
||||||
|
dupes += 1
|
||||||
|
c.commit(); c.close()
|
||||||
|
return jsonify(added=added, dupes=dupes)
|
||||||
|
|
||||||
|
@app.route("/api/subcount")
|
||||||
|
def api_subcount():
|
||||||
|
c = db()
|
||||||
|
n = c.execute("SELECT COUNT(*) n FROM subscribers WHERE status='active'").fetchone()["n"]
|
||||||
|
c.close()
|
||||||
|
return jsonify(active=n)
|
||||||
|
|
||||||
|
@app.route("/api/campaign", methods=["POST"])
|
||||||
|
def api_campaign():
|
||||||
|
d = request.json or {}
|
||||||
|
c = db()
|
||||||
|
cur = c.execute("INSERT INTO campaigns (name, subject, body, status, segment_interest) VALUES (?,?,?, 'ready', ?)",
|
||||||
|
(d.get("subject", "Campaign")[:40], d.get("subject", ""), d.get("body", ""),
|
||||||
|
(d.get("segment", "") or "").strip()))
|
||||||
|
cid = cur.lastrowid
|
||||||
|
c.commit(); c.close()
|
||||||
|
return jsonify(id=cid)
|
||||||
|
|
||||||
|
@app.route("/api/start", methods=["POST"])
|
||||||
|
def api_start():
|
||||||
|
cfg = load_config()
|
||||||
|
cfg["stopped"] = False
|
||||||
|
save_config(cfg)
|
||||||
|
c = db()
|
||||||
|
row = c.execute("SELECT id FROM campaigns ORDER BY id DESC LIMIT 1").fetchone()
|
||||||
|
c.close()
|
||||||
|
if not row:
|
||||||
|
return jsonify(ok=False, error="save a campaign first")
|
||||||
|
return jsonify(get_engine().run_campaign(row["id"]))
|
||||||
|
|
||||||
|
@app.route("/api/suppress", methods=["POST"])
|
||||||
|
def api_suppress():
|
||||||
|
text = (request.json or {}).get("text", "")
|
||||||
|
c = db()
|
||||||
|
n = 0
|
||||||
|
for raw in io.StringIO(text).readlines():
|
||||||
|
e = raw.strip().lower()
|
||||||
|
if "@" in e:
|
||||||
|
c.execute("INSERT OR IGNORE INTO suppression (email, reason) VALUES (?, 'import')", (e,))
|
||||||
|
n += 1
|
||||||
|
c.commit()
|
||||||
|
total = c.execute("SELECT COUNT(*) n FROM suppression").fetchone()["n"]
|
||||||
|
c.close()
|
||||||
|
return jsonify(added=n, total=total)
|
||||||
|
|
||||||
|
@app.route("/api/abstats")
|
||||||
|
def api_abstats():
|
||||||
|
c = db()
|
||||||
|
rows = c.execute("SELECT subject, sent, opened FROM ab_variants WHERE campaign_id=(SELECT MAX(id) FROM campaigns)").fetchall()
|
||||||
|
c.close()
|
||||||
|
return jsonify([dict(r) for r in rows])
|
||||||
|
|
||||||
|
@app.route("/unsub/<email>")
|
||||||
|
def unsub(email):
|
||||||
|
e = email.strip().lower()
|
||||||
|
if "@" not in e:
|
||||||
|
return "bad link", 400
|
||||||
|
c = db()
|
||||||
|
c.execute("INSERT OR IGNORE INTO suppression (email, reason) VALUES (?, 'unsubscribe')", (e,))
|
||||||
|
c.execute("UPDATE subscribers SET status='unsubscribed' WHERE email=?", (e,))
|
||||||
|
c.commit()
|
||||||
|
c.close()
|
||||||
|
return render_template_string("<body style='background:#0d0f1a;color:#e8eaf6;font-family:sans-serif;display:grid;place-items:center;height:100vh'><div style='text-align:center'><h1>✓</h1><p>You're unsubscribed. No hard feelings.</p></div></body>")
|
||||||
|
|
||||||
|
@app.route("/api/testsend", methods=["POST"])
|
||||||
|
def api_testsend():
|
||||||
|
d = request.json or {}
|
||||||
|
to_email = (d.get("email") or "").strip().lower()
|
||||||
|
if "@" not in to_email:
|
||||||
|
return jsonify(ok=False, error="bad email")
|
||||||
|
from engine import OllamaClient, render_template, render_subject, personalize_intro
|
||||||
|
cfg = load_config()
|
||||||
|
sub = {"first_name": d.get("first_name", "Jane"), "last_name": "",
|
||||||
|
"email": to_email, "interest": d.get("interest", "our updates")}
|
||||||
|
intro = ""
|
||||||
|
if cfg.get("personalize"):
|
||||||
|
oc = OllamaClient(cfg["ollama_url"], cfg["ollama_model"])
|
||||||
|
intro = personalize_intro(oc, sub, cfg)
|
||||||
|
c = db()
|
||||||
|
camp = c.execute("SELECT * FROM campaigns ORDER BY id DESC LIMIT 1").fetchone()
|
||||||
|
c.close()
|
||||||
|
subject = render_subject(camp["subject"] if camp else "Hello {{first_name}}", sub)
|
||||||
|
body = render_template(camp["body"] if camp else "<p>{{intro}}</p>", sub, cfg, intro)
|
||||||
|
try:
|
||||||
|
from engine import Sender
|
||||||
|
mid = Sender(cfg).send(to_email, subject, body)
|
||||||
|
return jsonify(ok=True, id=mid, intro=intro[:200])
|
||||||
|
except Exception as e:
|
||||||
|
return jsonify(ok=False, error=str(e)[:300])
|
||||||
|
|
||||||
|
@app.route("/api/stop", methods=["POST"])
|
||||||
|
def api_stop():
|
||||||
|
eng = get_engine()
|
||||||
|
eng.state["running"] = False
|
||||||
|
cfg = load_config(); cfg["stopped"] = True; save_config(cfg)
|
||||||
|
return jsonify(ok=True)
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
app.run(host="127.0.0.1", port=8899, threaded=True)
|
||||||
444
engine.py
Normal file
444
engine.py
Normal file
@@ -0,0 +1,444 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Mailer Engine — Ollama-personalized email campaign sender.
|
||||||
|
Runs on Windows (VPS). SQLite-backed. SES delivery. Ollama over Tailscale.
|
||||||
|
"""
|
||||||
|
import sqlite3, json, time, threading, queue, os, re, html, logging
|
||||||
|
from datetime import datetime
|
||||||
|
import requests
|
||||||
|
|
||||||
|
APP_DIR = os.path.dirname(os.path.abspath(__file__))
|
||||||
|
DB_PATH = os.path.join(APP_DIR, "mailer.db")
|
||||||
|
LOG_PATH = os.path.join(APP_DIR, "engine.log")
|
||||||
|
|
||||||
|
logging.basicConfig(filename=LOG_PATH, level=logging.INFO,
|
||||||
|
format="%(asctime)s %(levelname)s %(message)s")
|
||||||
|
log = logging.getLogger("mailer")
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ config
|
||||||
|
def load_config():
|
||||||
|
defaults = {
|
||||||
|
"ollama_url": "http://100.103.34.0:11440", # nightmare tailnet via non-chunked shim
|
||||||
|
"ollama_model": "mgraffam/gemma4-heretic:12b",
|
||||||
|
"ses_region": "us-east-1",
|
||||||
|
"send_mode": "smtp", # smtp (works now) | ses (needs AWS keys)
|
||||||
|
"smtp_host": "smtp.protonmail.ch",
|
||||||
|
"smtp_port": 587,
|
||||||
|
"smtp_user": "makemoneys8@proton.me",
|
||||||
|
"smtp_pass": "",
|
||||||
|
"from_name": "drjones",
|
||||||
|
"from_email": "",
|
||||||
|
"reply_to": "",
|
||||||
|
"rate_per_minute": 60, # SES default 14/s; keep low for reputation
|
||||||
|
"batch_size": 50,
|
||||||
|
"pause_between_batches_sec": 20,
|
||||||
|
"personalize": True,
|
||||||
|
"personalize_prompt": "Write a 2-3 sentence friendly, personal email greeting for {first_name}. They are a subscriber interested in {interest}. Warm, human, no hype, no exclamation marks. Sign as {from_name}.",
|
||||||
|
"unsubscribe_footer": True,
|
||||||
|
"send_hours": "9-18", # server-local hour window; outside = pause
|
||||||
|
"ab_subjects": True, # gemma4 writes 3 subject variants
|
||||||
|
"stopped": False,
|
||||||
|
}
|
||||||
|
c = sqlite3.connect(DB_PATH)
|
||||||
|
c.execute("CREATE TABLE IF NOT EXISTS config (k TEXT PRIMARY KEY, v TEXT)")
|
||||||
|
for k, v in defaults.items():
|
||||||
|
c.execute("INSERT OR IGNORE INTO config (k,v) VALUES (?,?)", (k, json.dumps(v)))
|
||||||
|
c.commit()
|
||||||
|
out = {k: json.loads(v) for (k, v) in c.execute("SELECT k,v FROM config")}
|
||||||
|
c.close()
|
||||||
|
return out
|
||||||
|
|
||||||
|
def save_config(cfg):
|
||||||
|
c = sqlite3.connect(DB_PATH)
|
||||||
|
for k, v in cfg.items():
|
||||||
|
c.execute("INSERT OR REPLACE INTO config (k,v) VALUES (?,?)", (k, json.dumps(v)))
|
||||||
|
c.commit(); c.close()
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ db
|
||||||
|
def db():
|
||||||
|
c = sqlite3.connect(DB_PATH, check_same_thread=False)
|
||||||
|
c.row_factory = sqlite3.Row
|
||||||
|
return c
|
||||||
|
|
||||||
|
def init_db():
|
||||||
|
c = db()
|
||||||
|
c.executescript("""
|
||||||
|
CREATE TABLE IF NOT EXISTS subscribers (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
email TEXT UNIQUE NOT NULL,
|
||||||
|
first_name TEXT DEFAULT '',
|
||||||
|
last_name TEXT DEFAULT '',
|
||||||
|
interest TEXT DEFAULT '',
|
||||||
|
status TEXT DEFAULT 'active', -- active|unsubscribed|bounced|complained
|
||||||
|
source TEXT DEFAULT 'import',
|
||||||
|
created_at TEXT DEFAULT (datetime('now'))
|
||||||
|
);
|
||||||
|
CREATE TABLE IF NOT EXISTS sends (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
subscriber_id INTEGER,
|
||||||
|
campaign_id INTEGER,
|
||||||
|
email TEXT,
|
||||||
|
subject TEXT,
|
||||||
|
body TEXT,
|
||||||
|
status TEXT DEFAULT 'queued', -- queued|personalizing|sent|failed|skipped
|
||||||
|
error TEXT DEFAULT '',
|
||||||
|
message_id TEXT DEFAULT '',
|
||||||
|
sent_at TEXT
|
||||||
|
);
|
||||||
|
CREATE TABLE IF NOT EXISTS campaigns (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
name TEXT,
|
||||||
|
subject TEXT,
|
||||||
|
body TEXT, -- HTML with {{first_name}} etc
|
||||||
|
created_at TEXT DEFAULT (datetime('now')),
|
||||||
|
status TEXT DEFAULT 'draft', -- draft|ready|running|paused|done
|
||||||
|
segment_interest TEXT DEFAULT ''
|
||||||
|
);
|
||||||
|
CREATE TABLE IF NOT EXISTS suppression (
|
||||||
|
email TEXT PRIMARY KEY,
|
||||||
|
reason TEXT DEFAULT 'manual',
|
||||||
|
created_at TEXT DEFAULT (datetime('now'))
|
||||||
|
);
|
||||||
|
CREATE TABLE IF NOT EXISTS ab_variants (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
campaign_id INTEGER,
|
||||||
|
subject TEXT,
|
||||||
|
sent INTEGER DEFAULT 0,
|
||||||
|
opened INTEGER DEFAULT 0
|
||||||
|
);
|
||||||
|
CREATE TABLE IF NOT EXISTS events (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
send_id INTEGER,
|
||||||
|
type TEXT, -- open|click|bounce|complaint|unsubscribe
|
||||||
|
ts TEXT DEFAULT (datetime('now'))
|
||||||
|
);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_sends_status ON sends(status);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_subs_status ON subscribers(status);
|
||||||
|
""")
|
||||||
|
c.commit(); c.close()
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ ollama
|
||||||
|
class OllamaClient:
|
||||||
|
def __init__(self, url, model, timeout=300):
|
||||||
|
self.url = url.rstrip("/")
|
||||||
|
self.model = model
|
||||||
|
self.timeout = timeout
|
||||||
|
self._fail_until = 0
|
||||||
|
|
||||||
|
def healthy(self):
|
||||||
|
try:
|
||||||
|
r = requests.get(f"{self.url}/api/tags", timeout=5)
|
||||||
|
return r.status_code == 200
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
|
||||||
|
def generate(self, prompt):
|
||||||
|
if time.time() < self._fail_until:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
r = requests.post(f"{self.url}/api/generate",
|
||||||
|
json={"model": self.model, "prompt": prompt, "stream": False,
|
||||||
|
"options": {"num_predict": 350, "temperature": 0.8}, "keep_alive": "15m"},
|
||||||
|
timeout=self.timeout)
|
||||||
|
r.raise_for_status()
|
||||||
|
d = r.json()
|
||||||
|
text = (d.get("response") or "").strip()
|
||||||
|
if not text:
|
||||||
|
text = (d.get("thinking") or "").strip()
|
||||||
|
# strip any raw-CoT
|
||||||
|
if "</think>" in text:
|
||||||
|
text = text.split("</think>")[-1]
|
||||||
|
return text.strip()
|
||||||
|
except Exception as e:
|
||||||
|
log.warning(f"ollama fail: {e}")
|
||||||
|
self._fail_until = time.time() + 120 # backoff 2 min
|
||||||
|
return None
|
||||||
|
|
||||||
|
def personalize_intro(ollama, sub, cfg):
|
||||||
|
if not cfg.get("personalize") or not ollama:
|
||||||
|
return ""
|
||||||
|
prompt = ("Reply with ONLY the greeting text itself — no plan, no options, no bullets, no thinking out loud.\n"
|
||||||
|
+ cfg["personalize_prompt"].format(
|
||||||
|
first_name=sub["first_name"] or "there",
|
||||||
|
interest=sub["interest"] or "our updates",
|
||||||
|
from_name=cfg.get("from_name", "")))
|
||||||
|
out = ollama.generate(prompt)
|
||||||
|
if not out:
|
||||||
|
return ""
|
||||||
|
# sanitize: keep it short, no subject lines the model might add
|
||||||
|
# gemma4-heretic wraps candidate greetings in "double quotes" inside planning bullets.
|
||||||
|
quoted = re.findall(r'"([^"\n]{25,400})"', out)
|
||||||
|
if quoted:
|
||||||
|
text = quoted[-1].strip()
|
||||||
|
else:
|
||||||
|
keep = []
|
||||||
|
for l in out.splitlines():
|
||||||
|
t = l.strip()
|
||||||
|
if not t:
|
||||||
|
continue
|
||||||
|
if t.startswith("*") or t.startswith("-"):
|
||||||
|
continue
|
||||||
|
low = t.lower()
|
||||||
|
if low.startswith(("constraint", "subject:", "tone:", "content:", "format",
|
||||||
|
"option", "plan", "context", "recipient", "interest",
|
||||||
|
"sign-off", "sign:")):
|
||||||
|
continue
|
||||||
|
keep.append(t)
|
||||||
|
text = " ".join(keep) if keep else out.strip()
|
||||||
|
return text[:600]
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ template
|
||||||
|
def render_template(body, sub, cfg, intro=""):
|
||||||
|
d = sub["first_name"] or "there"
|
||||||
|
out = body
|
||||||
|
for k, v in {
|
||||||
|
"{{first_name}}": d,
|
||||||
|
"{{first_name_or_there}}": d,
|
||||||
|
"{{last_name}}": sub["last_name"] or "",
|
||||||
|
"{{email}}": sub["email"],
|
||||||
|
"{{interest}}": sub["interest"] or "our updates",
|
||||||
|
"{{from_name}}": cfg.get("from_name", ""),
|
||||||
|
"{{intro}}": intro,
|
||||||
|
}.items():
|
||||||
|
out = out.replace(k, html.escape(v) if "{{intro}}" != k else v)
|
||||||
|
# also {first_name} single-brace style
|
||||||
|
out = re.sub(r"\{first_name\}", d, out)
|
||||||
|
if cfg.get("unsubscribe_footer"):
|
||||||
|
foot = ('<div style="margin-top:32px;padding-top:12px;border-top:1px solid #ddd;'
|
||||||
|
'font-size:12px;color:#999">You get this because you subscribed. '
|
||||||
|
'<a href="{{UNSUB}}">Unsubscribe</a></div>')
|
||||||
|
out = out + foot
|
||||||
|
return out
|
||||||
|
|
||||||
|
def render_subject(subject, sub):
|
||||||
|
out = subject.replace("{{first_name}}", sub["first_name"] or "there")
|
||||||
|
out = re.sub(r"\{first_name\}", sub["first_name"] or "there", out)
|
||||||
|
return out
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ sender
|
||||||
|
class Sender:
|
||||||
|
"""SES via boto3; lazy import so the GUI can start without creds."""
|
||||||
|
def __init__(self, cfg):
|
||||||
|
self.cfg = cfg
|
||||||
|
self._client = None
|
||||||
|
|
||||||
|
def client(self):
|
||||||
|
if self._client is None:
|
||||||
|
import boto3
|
||||||
|
self._client = boto3.client("sesv2", region_name=self.cfg.get("ses_region", "us-east-1"))
|
||||||
|
return self._client
|
||||||
|
|
||||||
|
def send(self, to_email, subject, html_body):
|
||||||
|
if self.cfg.get("send_mode") == "smtp":
|
||||||
|
return self._send_smtp(to_email, subject, html_body)
|
||||||
|
c = self.client()
|
||||||
|
resp = c.send_email(
|
||||||
|
FromEmailAddress=f'{self.cfg.get("from_name","")} <{self.cfg["from_email"]}>' if self.cfg.get("from_name") else self.cfg["from_email"],
|
||||||
|
Destination={"ToAddresses": [to_email]},
|
||||||
|
Content={
|
||||||
|
"Simple": {
|
||||||
|
"Subject": {"Data": subject, "Charset": "UTF-8"},
|
||||||
|
"Body": {"Html": {"Data": html_body, "Charset": "UTF-8"}},
|
||||||
|
}
|
||||||
|
},
|
||||||
|
ReplyToAddresses=[self.cfg["reply_to"]] if self.cfg.get("reply_to") else [],
|
||||||
|
)
|
||||||
|
return resp["MessageId"]
|
||||||
|
|
||||||
|
def _send_smtp(self, to_email, subject, html_body):
|
||||||
|
import smtplib
|
||||||
|
from email.mime.multipart import MIMEMultipart
|
||||||
|
from email.mime.text import MIMEText
|
||||||
|
msg = MIMEMultipart("alternative")
|
||||||
|
msg["Subject"] = subject
|
||||||
|
msg["From"] = f'{self.cfg.get("from_name","")} <{self.cfg.get("smtp_user","")}>'
|
||||||
|
msg["To"] = to_email
|
||||||
|
if self.cfg.get("reply_to"):
|
||||||
|
msg["Reply-To"] = self.cfg["reply_to"]
|
||||||
|
msg.attach(MIMEText(html_body, "html", "utf-8"))
|
||||||
|
with smtplib.SMTP(self.cfg.get("smtp_host", "smtp.mail.me.com"),
|
||||||
|
int(self.cfg.get("smtp_port", 587)), timeout=30) as srv:
|
||||||
|
srv.starttls()
|
||||||
|
srv.login(self.cfg.get("smtp_user", ""), self.cfg.get("smtp_pass", ""))
|
||||||
|
srv.send_message(msg)
|
||||||
|
return f"smtp-{int(time.time())}"
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ engine
|
||||||
|
class Engine:
|
||||||
|
def __init__(self):
|
||||||
|
init_db()
|
||||||
|
self.state = {"running": False, "current": "", "sent": 0, "failed": 0,
|
||||||
|
"skipped": 0, "total": 0, "started_at": ""}
|
||||||
|
self.thread = None
|
||||||
|
|
||||||
|
def status(self):
|
||||||
|
c = db()
|
||||||
|
counts = {r["status"]: r["n"] for r in c.execute(
|
||||||
|
"SELECT status, COUNT(*) n FROM sends GROUP BY status")}
|
||||||
|
camp = c.execute("SELECT * FROM campaigns ORDER BY id DESC LIMIT 1").fetchone()
|
||||||
|
c.close()
|
||||||
|
return {**self.state, "counts": counts,
|
||||||
|
"campaign": dict(camp) if camp else None}
|
||||||
|
|
||||||
|
def run_campaign(self, campaign_id):
|
||||||
|
if self.state["running"]:
|
||||||
|
return {"ok": False, "error": "already running"}
|
||||||
|
self.thread = threading.Thread(target=self._run, args=(campaign_id,), daemon=True)
|
||||||
|
self.thread.start()
|
||||||
|
return {"ok": True}
|
||||||
|
|
||||||
|
def _in_send_hours(self, cfg):
|
||||||
|
try:
|
||||||
|
lo, hi = cfg.get("send_hours", "9-18").split("-")
|
||||||
|
h = datetime.now().hour
|
||||||
|
return int(lo) <= h < int(hi)
|
||||||
|
except Exception:
|
||||||
|
return True
|
||||||
|
|
||||||
|
def _run(self, campaign_id):
|
||||||
|
cfg = load_config()
|
||||||
|
c = db()
|
||||||
|
camp = c.execute("SELECT * FROM campaigns WHERE id=?", (campaign_id,)).fetchone()
|
||||||
|
if not camp:
|
||||||
|
c.close(); return
|
||||||
|
# queue all active subscribers not yet sent for this campaign
|
||||||
|
seg = (camp["segment_interest"] or "").strip() if "segment_interest" in camp.keys() else ""
|
||||||
|
if seg:
|
||||||
|
rows = c.execute("""
|
||||||
|
SELECT s.* FROM subscribers s
|
||||||
|
WHERE s.status='active'
|
||||||
|
AND s.interest LIKE ?
|
||||||
|
AND s.id NOT IN (SELECT subscriber_id FROM sends WHERE campaign_id=? AND status IN ('sent','skipped'))
|
||||||
|
ORDER BY s.id
|
||||||
|
""", (f"%{seg}%", campaign_id)).fetchall()
|
||||||
|
else:
|
||||||
|
rows = c.execute("""
|
||||||
|
SELECT s.* FROM subscribers s
|
||||||
|
WHERE s.status='active'
|
||||||
|
AND s.id NOT IN (SELECT subscriber_id FROM sends WHERE campaign_id=? AND status IN ('sent','skipped'))
|
||||||
|
ORDER BY s.id
|
||||||
|
""", (campaign_id,)).fetchall()
|
||||||
|
for r in rows:
|
||||||
|
c.execute("INSERT OR IGNORE INTO sends (subscriber_id, campaign_id, email, status) VALUES (?,?,?,'queued')",
|
||||||
|
(r["id"], campaign_id, r["email"]))
|
||||||
|
c.commit()
|
||||||
|
self.state.update({"running": True, "sent": 0, "failed": 0, "skipped": 0,
|
||||||
|
"total": len(rows), "started_at": datetime.now().isoformat(timespec='seconds')})
|
||||||
|
log.info(f"campaign {campaign_id} start: {len(rows)} recipients")
|
||||||
|
|
||||||
|
ollama = OllamaClient(cfg["ollama_url"], cfg["ollama_model"]) if cfg.get("personalize") else None
|
||||||
|
if ollama:
|
||||||
|
self.state["current"] = f"checking Ollama at {cfg['ollama_url']}..."
|
||||||
|
if ollama.healthy():
|
||||||
|
log.info("ollama healthy")
|
||||||
|
else:
|
||||||
|
log.warning("ollama NOT reachable — sending without personalization")
|
||||||
|
self.state["current"] = "Ollama unreachable — sending generic (will retry it periodically)"
|
||||||
|
|
||||||
|
# A/B subject variants (feature 6): gemma4 writes 2 alternates; variant 0 = campaign subject
|
||||||
|
variants = [camp["subject"]]
|
||||||
|
if cfg.get("ab_subjects") and ollama and ollama.healthy():
|
||||||
|
for i in range(2):
|
||||||
|
alt = ollama.generate(
|
||||||
|
f"Rewrite this email subject line differently — same meaning, fresh angle, under 60 chars, no quotes. Reply with ONLY the subject.\nSubject: {camp['subject']}")
|
||||||
|
if alt:
|
||||||
|
alt = alt.strip().strip('"').splitlines()[0][:80]
|
||||||
|
if alt and alt.lower() not in [v.lower() for v in variants]:
|
||||||
|
variants.append(alt)
|
||||||
|
if len(variants) > 1:
|
||||||
|
c.execute("DELETE FROM ab_variants WHERE campaign_id=?", (campaign_id,))
|
||||||
|
for v in variants:
|
||||||
|
c.execute("INSERT INTO ab_variants (campaign_id, subject) VALUES (?,?)", (campaign_id, v))
|
||||||
|
c.commit()
|
||||||
|
log.info(f"AB variants: {variants}")
|
||||||
|
|
||||||
|
sender = Sender(cfg)
|
||||||
|
batch_n = 0
|
||||||
|
try:
|
||||||
|
sends = c.execute("SELECT * FROM sends WHERE campaign_id=? AND status='queued' ORDER BY id",
|
||||||
|
(campaign_id,)).fetchall()
|
||||||
|
for snd in sends:
|
||||||
|
if cfg.get("stopped") or not self.state["running"]:
|
||||||
|
log.info("stopped by operator")
|
||||||
|
break
|
||||||
|
sub = c.execute("SELECT * FROM subscribers WHERE id=?", (snd["subscriber_id"],)).fetchone()
|
||||||
|
if not sub or sub["status"] != "active":
|
||||||
|
c.execute("UPDATE sends SET status='skipped' WHERE id=?", (snd["id"],)); c.commit()
|
||||||
|
self.state["skipped"] += 1
|
||||||
|
continue
|
||||||
|
if c.execute("SELECT 1 FROM suppression WHERE email=?", (sub["email"],)).fetchone():
|
||||||
|
c.execute("UPDATE sends SET status='skipped', error='suppressed' WHERE id=?", (snd["id"],)); c.commit()
|
||||||
|
self.state["skipped"] += 1
|
||||||
|
continue
|
||||||
|
if not self._in_send_hours(cfg):
|
||||||
|
# outside sending window: pause loop (re-check every 5 min)
|
||||||
|
self.state["current"] = f"outside send hours ({cfg.get('send_hours')}) — sleeping"
|
||||||
|
c.execute("UPDATE sends SET status='queued' WHERE id=?", (snd["id"],)); c.commit()
|
||||||
|
time.sleep(300)
|
||||||
|
continue
|
||||||
|
self.state["current"] = sub["email"]
|
||||||
|
# personalize
|
||||||
|
intro = ""
|
||||||
|
if ollama and cfg.get("personalize"):
|
||||||
|
c.execute("UPDATE sends SET status='personalizing' WHERE id=?", (snd["id"],)); c.commit()
|
||||||
|
intro = personalize_intro(ollama, sub, cfg)
|
||||||
|
variant_i = self.state["sent"] % len(variants)
|
||||||
|
subject = render_subject(variants[variant_i], sub)
|
||||||
|
body = render_template(camp["body"], sub, cfg, intro)
|
||||||
|
# send
|
||||||
|
try:
|
||||||
|
mid = sender.send(sub["email"], subject, body)
|
||||||
|
c.execute("UPDATE sends SET status='sent', subject=?, body=?, message_id=?, sent_at=datetime('now') WHERE id=?",
|
||||||
|
(subject, body, mid, snd["id"]))
|
||||||
|
if len(variants) > 1:
|
||||||
|
c.execute("UPDATE ab_variants SET sent=sent+1 WHERE campaign_id=? AND subject=?",
|
||||||
|
(campaign_id, variants[variant_i]))
|
||||||
|
self.state["sent"] += 1
|
||||||
|
except Exception as e:
|
||||||
|
err = str(e)[:300]
|
||||||
|
log.error(f"send fail {sub['email']}: {err}")
|
||||||
|
c.execute("UPDATE sends SET status='failed', error=? WHERE id=?", (err, snd["id"]))
|
||||||
|
self.state["failed"] += 1
|
||||||
|
if "Throttling" in err:
|
||||||
|
time.sleep(30)
|
||||||
|
c.commit()
|
||||||
|
batch_n += 1
|
||||||
|
time.sleep(60.0 / max(1, cfg.get("rate_per_minute", 60)))
|
||||||
|
if batch_n >= cfg.get("batch_size", 50):
|
||||||
|
batch_n = 0
|
||||||
|
self.state["current"] = f"batch done — pausing {cfg.get('pause_between_batches_sec',20)}s"
|
||||||
|
time.sleep(cfg.get("pause_between_batches_sec", 20))
|
||||||
|
finally:
|
||||||
|
self.state["running"] = False
|
||||||
|
self.state["current"] = "idle"
|
||||||
|
c.execute("UPDATE campaigns SET status='done' WHERE id=?", (campaign_id,))
|
||||||
|
c.commit(); c.close()
|
||||||
|
log.info(f"campaign {campaign_id} done: sent={self.state['sent']} failed={self.state['failed']}")
|
||||||
|
|
||||||
|
ENGINE = None
|
||||||
|
def get_engine():
|
||||||
|
global ENGINE
|
||||||
|
if ENGINE is None:
|
||||||
|
ENGINE = Engine()
|
||||||
|
ENGINE.auto_resume()
|
||||||
|
return ENGINE
|
||||||
|
|
||||||
|
def auto_resume_thread():
|
||||||
|
"""Called from app startup: find campaign marked running/ready with queued sends and restart."""
|
||||||
|
import time as _t
|
||||||
|
_t.sleep(15) # let the web app bind first
|
||||||
|
try:
|
||||||
|
eng = get_engine()
|
||||||
|
c = db()
|
||||||
|
row = c.execute("SELECT id FROM campaigns WHERE status IN ('running','ready') ORDER BY id DESC LIMIT 1").fetchone()
|
||||||
|
n = c.execute("SELECT COUNT(*) n FROM sends WHERE status='queued'").fetchone()["n"]
|
||||||
|
c.close()
|
||||||
|
if row and n and not eng.state["running"]:
|
||||||
|
log.info(f"auto-resume campaign {row['id']} ({n} queued)")
|
||||||
|
eng.run_campaign(row["id"])
|
||||||
|
except Exception as e:
|
||||||
|
log.error(f"auto-resume failed: {e}")
|
||||||
|
|
||||||
|
import threading as _threading
|
||||||
|
def start_auto_resume():
|
||||||
|
_threading.Thread(target=auto_resume_thread, daemon=True).start()
|
||||||
42
ollama_shim.py
Normal file
42
ollama_shim.py
Normal file
@@ -0,0 +1,42 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Ollama shim for nightmare — binds 100.103.34.0:11440 (tailnet only), proxies /api/generate
|
||||||
|
to localhost:11434 and returns ONE plain, non-chunked response. Binds tailnet IP only so no
|
||||||
|
new exposure beyond the tailnet."""
|
||||||
|
import json
|
||||||
|
import http.server, socketserver, urllib.request
|
||||||
|
|
||||||
|
LOCAL = "http://127.0.0.1:11434"
|
||||||
|
BIND = ("100.103.34.0", 11440)
|
||||||
|
|
||||||
|
class H(http.server.BaseHTTPRequestHandler):
|
||||||
|
protocol_version = "HTTP/1.0" # no keep-alive, no chunked
|
||||||
|
def do_GET(self):
|
||||||
|
if self.path == "/api/tags":
|
||||||
|
body = urllib.request.urlopen(LOCAL + "/api/tags", timeout=10).read()
|
||||||
|
self.send_response(200); self.send_header("Content-Length", str(len(body)))
|
||||||
|
self.end_headers(); self.wfile.write(body)
|
||||||
|
else:
|
||||||
|
self.send_response(404); self.end_headers()
|
||||||
|
def do_POST(self):
|
||||||
|
n = int(self.headers.get("Content-Length", 0))
|
||||||
|
payload = json.loads(self.rfile.read(n) or b"{}")
|
||||||
|
payload["stream"] = False
|
||||||
|
req = urllib.request.Request(LOCAL + "/api/generate",
|
||||||
|
data=json.dumps(payload).encode(),
|
||||||
|
headers={"Content-Type": "application/json"})
|
||||||
|
with urllib.request.urlopen(req, timeout=600) as r:
|
||||||
|
body = r.read()
|
||||||
|
self.send_response(200)
|
||||||
|
self.send_header("Content-Type", "application/json")
|
||||||
|
self.send_header("Content-Length", str(len(body)))
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(body)
|
||||||
|
def log_message(self, *a):
|
||||||
|
pass
|
||||||
|
|
||||||
|
class TS(socketserver.ThreadingTCPServer):
|
||||||
|
allow_reuse_address = True
|
||||||
|
daemon_threads = True
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
TS(BIND, H).serve_forever()
|
||||||
9
start.bat
Normal file
9
start.bat
Normal file
@@ -0,0 +1,9 @@
|
|||||||
|
@echo off
|
||||||
|
title Postal Mailer
|
||||||
|
cd /d C:\Mailer
|
||||||
|
if not exist venv (
|
||||||
|
python -m venv venv
|
||||||
|
venv\Scripts\pip install --quiet flask requests boto3
|
||||||
|
)
|
||||||
|
venv\Scripts\python app.py
|
||||||
|
start http://127.0.0.1:8899
|
||||||
3
watchdog.bat
Normal file
3
watchdog.bat
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
@echo off
|
||||||
|
REM Postal watchdog: if :8899 not answering, force-restart the app
|
||||||
|
powershell -NoProfile -Command "$ok=$false; try { $r=Invoke-WebRequest -Uri 'http://127.0.0.1:8899/' -UseBasicParsing -TimeoutSec 8; $ok=($r.StatusCode -eq 200) } catch {}; if (-not $ok) { Get-Process python* -ErrorAction SilentlyContinue | Stop-Process -Force; schtasks /run /tn MailerApp }"
|
||||||
Reference in New Issue
Block a user