Project: mlist - The Ruby Toolbox

An insane attempt to build a mail list server library that can be used in other
applications very easily. The first target is Ruby applications that can load
MList models for direct, embedded integration. It will later have a RESTful API
so that any language/architecture can be easily integrated. That may depend
heavily on community involvement...

== FEATURES/PROBLEMS:

If you have any experience with mailing lists, things can be understood most
quickly with this: MList is the mailing list server and database, your
application is the list manager.

MList is being used in a production environment as a Ruby gem/Rails plugin. I
decided not to use other list software primarily because a) we already had a
running, sophisticated list manager that I didn't care to synchronize with
something else, b) we have an exceptional UI design for listing messages in
threads and ultimately will have search and c) integration options for the other
software looked to be a problem in themselves.

And then there's the fame and glory that comes with building something like
this...

There is a LOT to do: segmenting, i18n, backscatter - only the Mailman developers
know what else. I have enough experience to know that rewrites are NEVER as easy
as they seem. Alas, I go boldly forward.

==== Thread Trees

A Thread answers a tree of messages where each message in the tree knows it's
parent, children, previous and next message, and whether it is the root or a leaf
in the tree. The messages are actually delegates, wrappers around the real
message instances from the thread.

This approach makes a few assumptions:

# You want the tree because you will be displaying it # Moving through a tree
message by message should 'feel' right; next is determined by walking the tree
depth first.

It may or may not prove useful someday to have this knowledge in the form of
something like awesome_nested_set.

==== Extracting 'from' IP address from emails is not implemented

http://compnetworking.about.com/od/workingwithipaddresses/qt/ipaddressemail.htm

==== Deleting messages

Mail list servers don't typically deal with this, do they? When an email is sent,
it's sent! If you delete one, how can a reply to it be accurately injected into a
partially broken tree?

Since MList is designed to integrate with applications that want to distribute
messages, those applications may want to delete messages. I feel, at the time of
writing this, that the feature of deleting is NOT a problem that MList should
attempt to address in great detail. The models are ActiveRecord descendants, so
they can be deleted with #destroy, et. al. MList will not prevent that from
happening. This has implications, of course.

# MList::Thread#tree will likely break if messages are gone.

In my own usage, I have opted for adding a column to my mlist_messages table,
deleted_at. The application will make decisions based on whether that column has
a value or not. I would suggest you implement something similar, always favoring
leaving the records in place (paranoid delete). Since MList::MailList#messages
(and other such has_many associations) don't know about this column, they will
always answer collections which still include those messages.

When an MList::MailList is destroyed, all it's MList::Messages and MList::Threads
will be deleted (:dependent => :delete_all). If no other MList::Messages are
referencing the MList::Email records, they will also be deleted.

==== Ensuring Delivery

The internet email system is a mess. Spam has proven to be a significant
problem which plagues both your inbox and the service providers who work to
deliver email into it. So, what must you, the user of MList do, to ensure that
your software plays nice, and your emails are delivered?

Of course, this problem goes well beyond MList - any email you send from your
application servers will suffer the consequences of a misconfigured domain.
Here's what I've learned:

* Go to http://old.openspf.org/wizard.html and create your SPF record
content, to be placed in a TXT record for your domain.
- If you are using Google Apps, check this out:
http://www.google.com/support/a/bin/answer.py?hl=en&answer=33786
* Make sure you have an abuse@yourdomain.com and postmaster@yourdomain.com
address.
- If you are using Google Apps, you will need to create 'groups' for those
addresses, adding yourself as a recipient. You can read more about it
here: http://blog.wordtothewise.com/2009/01/google-apps-wheres-my-abuse/
* Be prepared to visit the Feedback Loop (FBL) configuration pages of many
ISPs, like the one at AOL: http://postmaster.aol.com/fbl/
* You'll probably want to have a way to resolve an email address (subscriber)
to an IP address from which the subscriber subscribed. This will help your
case should you have an ISP rejecting your mail.

There is a great article here:

http://community-support.engineyard.com/faqs/guides/making-sure-your-email-gets-delivered

Although that is EY focused, it gives you a good idea of some of the things
you should get right.

== SYNOPSIS:

Let's say you want your web application to have a mailing list feature. Let's
also say you care about the UI, and you don't want to learn all about creating
the correct HTML structures for a mailing list. You want to have lots of power
for searching the mail, and you have your own strategy for managing the lists.
You love Ruby. You want MList.

== REQUIREMENTS:

You'll need some gems.

* hpricot
* uuid (macaddr also)
* tmail
* active_support
* active_record

== INSTALL:

- gem sources add http://gemcutter.org
- sudo gem install mlist

For now, copy the mlist_ tables from the spec/fixtures/schema.rb file and move
them to a new migration in your application. Run the migration.

Now you'll need to create the MList::Server and provide it with a list manager
and MList::EmailServer::Default instance. Something like this in your
environment.rb after the initialize block (our gem needs to have been loaded):

Rails::Initializer.run do |config|
# Please do specify a version, and check for the latest!
config.gem "mlist", :version => '0.1.0'
end

MLIST_SERVER = MList::Server.new(
:list_manager => MList::Manager::Database.new,
:email_server => MList::EmailServer::Default.new(
MList::EmailServer::Pop.new(
:ssl => false,
:server => 'pop.gmail.com',
:port => '995',
:username => 'yourusername',
:password => 'yourpassword'
),
MList::EmailServer::Smtp.new(
ActionMailer::Base.smtp_settings # probably good enough!
)
)
)

Your list manager needs to implement only two methods. Check out (and use if you
like) the MList::Manager::Database for more information.

You'll need something to trigger the incoming server process. Take your pick from
http://wiki.rubyonrails.org/rails/pages/HowToRunBackgroundJobsInRails. In the
end, if you don't write your own incoming server thing and go with the POP GMail
example above, your background process will "MLIST_SERVER.email_server.execute".

Take a look at MList::EmailPost if you're building a UI. It can be posted to a
list something like this:

class MyList # instances of these are given by your list manager
def post(attributes)
email = MList::EmailPost.new({
:mailer => 'MyApplication'
}.merge(attributes))
raise 'You can validate these' unless email.valid?
message = MLIST_SERVER.mail_list(self).post(email)
# do what you will with message. it's already saved.
end
end

== LICENSE:

(The MIT License)

Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the 'Software'), to deal in the
Software without restriction, including without limitation the rights to use,
copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.