The 2600hz project, The new sound of open source telephony

Concurrency and reliability are baked into Erlang in a way that is different than most other languages I have worked with. This leads developers to think differently about application architecture, especially when it comes to error handling and failures within a large, distributed system.

Many of today’s most interesting problems in software development are also the most challenging. Concurrency in a multicore world and reliability when applications are growing ever larger and ever more complex must rank up there, along with naming and cache invalidation, as some of the most challenging problems in modern software development. Erlang is a pragmatic programming language aimed squarely at those problems. These traits make Erlang an excellent choice for building backend systems and APIs. For Example, the Riak and CouchDB databases are both implemented in Erlang as well as the RabbitMQ message queue. The Webmachine library makes writing RESTful APIs a snap.

A great post by Rackspace on Erlang that hits the nail on the head. We have to say, we agree. 

Today’s Hacker Story guest post is written by developer Michael Phillips.  

I’ve been spending a lot of time on the 2600hz platform lately, and it’s truly something to be excited about. Using their API you can leverage most features you’ve come to expect from a carrier grade phone system in a simple RESTful API. One of which is voicemail.

We all know the annoyances of traditional voicemail. If you had three new messages you had to listen to all three…in order…choosing to delete or save. Your carrier had voicemail box limits, and would purge messages older than 20 days. Sometimes your voicemail box was full, and no one else could leave a message. You had to use your phone to check, usually being charged for minutes. Compound all this with the fact that providers still do not support caller ID name.

Enter Google Voice

Most companies will charge for visual voicemail (Verizon customer over here). In case you don’t know, visual voicemail takes your traditional voicemail box and puts it into a email inbox format. You can see who it’s from, the time it was left, and the duration of the voicemail all at a glance.

Google Voice allows you to do this for free on an Android and iOS devices and they have a website you can use to manage your voicemails too. Google Voice is a great product. If you are simply looking to have Google handle your voice message, they are a simple solution. There are some scenarios, however, where we might want to host our own voicemail service, 2600hz makes this possible.

The 2600hz Approach

I was looking for tighter integration with my distributed PBX. I wanted one voicemail for my SIP devices and my cell phone. Technically you could do this with Google Voice, but there are a lot of limitations. With 2600hz you have a lot more flexibility because you control your DIDs, ring groups and voicemail boxes. In this respect you can design more complex dial plans and even use time of day routing!

In addition, Google Voice definitely can’t have your message waiting indicator convey you have waiting message. After I reprogrammed my cell phone to the 2600hz voicemail box, the instant someone left a message, my desktop phone would start to blink letting me know there was a message. At this point I could use my cellphone (via email), deskphone, or the custom 2600hz Winkstart interface via the web to check my message.

To accomplish this I setup a voicemail box, and enabled the voicemail to email option in the Winkstart GUI. I then selected a DID that I am going to dedicate to this voicemail box. Why, you might ask? Because I want the voicemail to pick up immediately. On Google Voice, at least on Verizon, the system doesn’t understand there is an incoming voicemail. As a result the call rings until the carrier voicemail attempts to pick up, then it forwards the call to Google where it will ring again as it waits for the voicemail to pick up there. A constant complaint I got from friends and family was “your voicemail took like 20 rings to pick up.” They usually just hung up in frustration. If I call this DID directly, I go right to voicemail. In fact, I programmed my speed dial for 1 to be this DID, giving some voicemail nostalgia.

Reprogramming your Voicemail by Carrier

Verizon
*7 + 11 digit DID
AT&T
*004) + 11 digit DID + #
Sprint
*28 + 10 digit DID (exclude 1 prefix)
T-Mobile
*004* + 11 digit DID + #
US Cellular
*74 + 10 digit DID (exclude 1 prefix)

Advantages

• Not hosted on Google. Their privacy policy gives them quite a bit of access to your usage information. WIth 2600z, we are on a private cloud. 
• Message Waiting Indicator (MWI) will light up on your desk phone!
• You can write your own voicemail interface or integrate it into existing applications
• You can export all your data

Disadvantages

• No VM to text - this was never useful to me anyway since the translation rarely made any sense 
• No mobile app. This will likely change soon. Using Crossbar and Phonegap it should be possible to have a 2600hz VM app for iOS and Android with little effort.
• Potential Cost. DIDs cost $1 per month. But that’s it, 12 bucks a year! 

You can reach Michael Phillips on Twitter at @NSRInvesting or at his website. 

This is how porting numbers happens at 2600hz, a poem to help you sung to the tune of a song you may recognize… 

My motto’s always been when it’s right it’s right
Why wait until the middle of the cold dark night?
Porting’s always gonna work better in advance
Bandwidth’s really quick so now’s your chance
Sales are really kickin up as of late
Take all of your port requests over to Kate
Phone numbers in flight!  Kate porting delight!

In our previous post, we talked about the differences between OpenSIPs and FreeSWITCH for handling call delivery. We will continue with that topic today. 

Now that we’ve understood the differences between FreeSWITCH and OpenSIPs we’ll take a deeper dive into the technical implementation features available in FreeSWITCH to support OpenSIPs (or any third-party proxy) as a front-end.

FreeSWITCH Support for OpenSIPs (or any proxy)
FreeSWITCH, in its quest to integrate with anything, has done a fairly good (and quickly approaching excellent) job at allowing integration with OpenSIPs. By default, if you route packets from an OpenSIPs box to FreeSWITCH, FreeSWITCH will analyze the headers in the SIP packet and figure out that the packet started somewhere else. Then, responses will automatically be directed back in the same direction they came in, which is incredibly useful for a variety of scenarios (we’ll cover these in next week’s post). In addition, IP-based authentication features are smart enough in FreeSWITCH to substitute the original IP address for the IP address of OpenSIPs on inbound packets.

Specifically, FreeSWITCH natively supports the following features:

X-Auth-IP Headers
In most cases FreeSWITCH needs to know the original source IP address of the SIP packet that is coming to it. Since these packets first hit OpenSIPs and then FreeSWITCH, this information is normally lost. To solve this problem, OpenSIPs can be programmed to add the original source IP address in a custom header named X-Auth-IP. It looks like this: X-Auth-IP: 74.23.22.11

When added, FreeSWITCH will automatically identify that the original IP address of the packet is 74.23.22.11. All security/access control checks and subsequent IP address information will be recorded using this source IP address.

Support for ACLs that identify proxies
FreeSWITCH supports the ability to accept packets and identify them from a SIP proxy. When used in conjunction with X-Auth-IP header mentioned above, this restricts who is allowed to submit a packet with X-Auth-IP enabled. This is an important protective measure because otherwise anyone could send a packet and fake the X-Auth-IP header and trick FreeSWITCH into thinking the source IP address was different. This way, you define where your OpenSIPs boxes are and then FreeSWITCH will only process X-Auth-IP headers from those boxes.

Support for Path Headers
The path header is assigned by OpenSIPs which allows FreeSWITCH to recognize, record and then save that header to know how FreeSWITCH should first route that call when inviting the device that has registered. As shown in the image below, when there is an established path header, FreeSWITCH must first route the call through OpenSIPs to invite that registered device. 

Support for Proxies During Registrations
FreeSWITCH automatically adds the ;fs_path=2.3.4.5 parameter when a phone registers. This parameter basically says “here’s the path to take in order to get back to this device.” This allows the packet to traverse back through the same proxy that the registration was originally received on. This is extremely important as it allows one or more FreeSWITCH boxes to contact a device via OpenSIPs and the device will always see the packets as coming from the same IP address. This helps handle a tremendous number of [Network Address Translation] NAT-related issues for you transparently.

Support for Originating Calls via Proxies
FreeSWITCH supports passing the ;fs_path=2.3.4.5 parameter manually via originate strings to Sofia. This parameter instructs Sofia to route calls via a proxy (like OpenSIPs). Unlike registrations (mentioned above) which automatically add this header, adding it manually lets you explicitly dictate when OpenSIPs should be used and if you have more than one server, it dictates which OpenSIPS server should be used when.

What’s Lacking in FreeSWITCH (and what we’re patching and contributing back!)
FreeSWITCH has done a great job with basic call flows in and out of OpenSIPs, but when it comes to SUBSCRIBE, NOTIFY, OPTIONS and other SIP messages, there is still much to be desired. Already in the FreeSWITCH tree (link) you will see some contributions from our team that fix these issues, with more on the way.

Specifically, the C code doesn’t always properly format proxy information for the sofia (SIP) library and sometimes packets get ignored or dropped on response even though they are properly received in FreeSWITCH. This is currently the case for OPTIONS, NOTIFY and some other packets. But don’t fret – pending patches will easily fix this and 2600hz is working on automated test routines to ensure this functionality doesn’t break in the future, even as improvements and new integrations are introduced.

To Sum it all Up
In Summary, OpenSIPs and FreeSWITCH make a killer combination. It takes some work to get things working perfectly, but ultimately the two serve different complementary roles that allow you to build extremely scalable telecom systems. We encourage folks trying to scale their platforms to become skilled in both platforms.

Google’s foray into the Voice market has made a mess of text messaging. Don’t believe us? Check out this laundry list of complaints that go well into the hundreds: 

Here. Here. Here. Here. Here. And Here. 

Or just search the web for “Can’t receive text messages from Google Voice” or “Can’t send text messages using Sprint and Google Voice.”  

What’s going on here? Why can’t the uber-search engine make something as mundane as text messaging work properly?

Google Voice Isn’t a Phone Company
The first thing to clear up is this — Google Voice is NOT a phone company. In fact, they’re not even close. They buy and sell minutes and phone numbers which are actually provided by companies like bandwidth.com (some Google Voice numbers will be listed as owned by Level3, since bandwidth.com also buys from Level3.)  **Fun tip: see www.numberguru.com to find out who owns your number.

In order to achieve this integration, Google Voice has likely worked with bandwidth.com to create software that keeps track of who owns what numbers and when. Likely the same integration now exists with Sprint, since you can use your Sprint number and integrate it with Google Voice.

How does this all really work? Say hello to the number inventory system. Number inventory systems keep track of what numbers are owned by whom for the purpose of routing calls “on-net” or “off-net.” An on-net call means you’re calling someone within the same service provider – for example, a Google Voice subscriber calling another Google Voice subscriber. An off-net call means you are calling someone at a different company – for example, a Google Voice subscriber calling to Sprint.

Since we are visual beings - here are some images to illustrate what this looks like. 

If a caller calls or texts another number within the same network, the call is designed to stay within that particular network to save money. So if one Google Voice user calls another Google Voice user, the call most likely won’t leave the Google network at all. But if you call a non-Google Voice number, that call hits bandwidth.com, who checks if they own the number. If they don’t, bandwidth.com passes the call onto Sprint or Verizon or AT&T or whoever does.

This would all work just fine if number ownership remained permanent. But number ownership is NOT permanent, and that’s where the problems ensue. 

Number Porting
In 1997 the FCC passed the Local Number Portability (LNP) act, requiring phone companies to allow you to move, or “port”, your number from one phone company to another. The idea was simple — let people keep their phone number and move it to another carrier who is offering better service or better deals, at any time.

So what really happens when you port your phone number? Well, without getting too technical, the process is theoretically simple – your phone number gets removed from one phone company’s number inventory database and placed into the database of another phone company.

But with Google Voice (and many other providers), this causes a big problem. Since Google Voice doesn’t actually own the phone number they don’t always get notified when you’ve moved your number away from them. This leads to your number possibly lying around in multiple databases.

Let’s take the example below and assume your number is (212) 555-1234. You’ve just recently ported your number from Google Voice to Sprint, but something has gone wrong and Google Voice hasn’t removed your number from their database:

In this scenario, people who have phones on the Sprint network will be able to call you because they are calling Sprint to Sprint and the Sprint database is the first checkpoint. But Google Voice callers? They won’t be able to reach you because Google thinks they’re still the owner of that number. Even worse, those folks might get your old voicemail (still active on Google Voice), in which case they’ll think they are in fact reaching you, and you’re just ignoring their calls! 

And it gets even worse…

SMS Overlays on Phone Numbers
Most people don’t know that SMS is actually a service that is often transmitted on a different network then voice calls. Interconnect companies, like Syniverse and TNSi keep databases of which phone companies own which phone numbers, strictly for the purpose of routing and optimizing text (SMS / MMS) message delivery. These databases are independent from the numbers that track your phone number for voice calls, adding another layer of complexity. It is this scenario that causes the most frustration – because phone calls between two people might be working just fine, but text messages do not. It seems random that this occurs, but it in fact is not. 

Let’s assume that porting (212) 555-1234 and your number properly got switched to Sprint’s database to make all voice calls work just fine. But check out the Syniverse database (which tracks which carrier owns a number for the purposes of text messaging) — it’s wrong!

In the above picture, you can see that Syniverse incorrectly shows the (212) 555-1234 number is owned by Google Voice.

The Plot Thickens
The above scenario used to be fairly uncommon, but the problem has become even more severe as the number of layers between the databases has increased. Remember how earlier we mentioned that Google Voice doesn’t own their numbers, bandwidth.com does? Well guess what that means. You guessed it,  another database! 

In the above scenario, bandwidth.com has the bad database entry. What does this mean? It’s likely that Sprint, Verizon and AT&T will be able to send text messages to the Sprint customer at (212) 555-1234, but Google Voice subscribers won’t be able to. To make matter worse, Google Voice subscribers will be able to call the Sprint number, but they won’t be able to text it. Frustrating and confusing!

So What Do You Do?
We think it’s time for some regulation here. Currently, there’s no reliable or formal process for asking a carrier to check their number database and make sure it’s correct. In addition, most customers are completely unaware of the issue at hand and blame the new carrier they’ve moved their number to for not working, when infact the problem is that the old carrier has not properly released the number! And of course, there’s no incentive for carriers to fix this.

That said, if you’re willing to do the legwork you can usually call these carriers and explain the situation, if you know the right folks to contact. 

Google – You can post your complaint to their forums, which seem to be monitored by the Google Voice team. One such extremely long thread is here. These folks all ported away from Google and asked to get their numbers removed. For some, it worked. For others, not so much.

Sprint Porting Department: (877) 688-1969
Verizon Wireless Porting: (877) 567-4899
AT&T Wireless Porting: (866) 895-1097

Ask the porting department to file and escalate a ticket to have your number properly provisioned in their system and removed from any other systems. Sometimes they’ll help, sometimes they won’t, but it’s worth a try.

A Few Notes
Nothing you do on your phone will actually fix this, regardless of how many times you “reset” your phone. You usually need to have your issue escalated.

Most companies have folks on the first-line support when you call who’ve never heard of Syniverse or TNSi and won’t know what you’re talking about. Make sure you get your ticket escalated - only the folks who do porting all day long have typically heard of this issue and know how to resolve it.

If you ported a landline or VoIP number to/from Google Voice or to/from your cell phone carrier, you may also experience the problems above since some landline/VoIP carriers are now slowly enabling SMS on their phones also. You’ll want to call your old carrier to make sure they’ve properly removed your number.

What Else Can Be Done?
This problem is technically quite solvable. The first thing carriers could do is come up with better strategies and be more open about the ability to look up number ownership. Most customers would figure out quickly if the number ownership was incorrect. In addition, it should be possible to automatically  spot-check number databases by comparing them with lookups on the SS7 (phone) network or via Syniverse and TNSi. One dirty little secret of these SMS networks (and why the problem is so prevalent) is that companies like Syniverse actually require you to upload a spreadsheet or CSV file of the phone numbers you own on a regular basis – a process that is horribly error prone as it’s often done by poorly written software or, worse, by humans.

In addition, it’s probably time for some regulation here. Carriers have no incentive to take responsibility for ensuring number porting happens correctly on both SMS and Voice networks together, and have no proper escalation procedures. That said, most consumers remain unaware that you can always complain to your state’s Public Utilities Commission who has “back-door access” to management at nearly every phone company. Just google your state and “Public Utilities Commission Complaints” and you’ll find a webpage where you can submit a formal complaint about a phone company. This strategy is generally very effective as the phone companies can get in trouble or even fined if they get too many complaints or caught doing something they’re not allowed to.

All in all, SMS, Voice and porting is all a huge cyclical mess. Try to avoid it if you can, or make sure you have someone experienced doing the ports who knows how to handle these scenarios if they occur.

PSTN

Short for Public Switched Telephone Network, basically what preceded the VoIP era. It’s your standard telephone network that involves telephone lines, fiber optic cables, cell networks, satellites and telephone cables that were dropped in the bottom of the ocean to help us connect and make calls overseas. Sometimes also referred to as POTS (Plain Old Telephone System) which is straightforward enough. 

DSL
Remember the days of AOL when going on the Internet meant you couldn’t make or answer any phone calls? DSL, which is short for Digital Subscriber Line allows you to use a high-speed internet connection that still used the same wires as a regular telephone line. 

Cloud
People love to throw around the term ‘Cloud’ to make it sound like the most innovative thing since sliced bread (which we’ve griped about before). But what it really means is that data doesn’t need to be stored on clunky hardware all around your office. It’s hosted on a separate server that you don’t need to ever see or give up space for. In just the same way that Apple offers iCloud so you can store all your data on a separate off-site server that you never have to see, and Dropbox allows you to share and store documents in a “drawer” you can access from anywhere,  your telecom data (call flow and switching, VoIP network can all continue as usual, except on servers that are off-site. 

Virtual PRI
Previously, telecom companies (AKA telcos)  would take individual T1s and create individual pipes, running at 64 kb to run 23 call (PRIs). This would obviously pose problems in scalability and get expensive pretty quickly. With Virtual PRIs, you can now take even 100 times that capacity to create individual T1s from within the pipe which would provide with less hardware and still provide the ‘virtualized’ PRI to run the call.  

Connection between DIDS and CLECs
CLEC is short for Competitive Local Exchange Carrier which refers to a carrier who is competing with another carrier that’s been competing in the local telephone market for awhile (Incumbent Local Exchange Carrier). CLECs are encouraged by the FCC to prevent monopolies from happening which means they’ve done a lot of paperwork to register but they gain privileged access that other unregistered CLEC carriers do not have. An example of a CLEC would be Bandwidth.com, while an ILEC would be someone like Level3 or AT&T. How does this relate to DIDs?

One of the aforementioned privileges that CLECs have are low-level access and control to DIDs. DIDs are Direct Inward Dials that basically give you an individual phone number. One of the easiest ways to think about it is this: consider your office’s set of numbers as a group. The last 4 digits of your own office number that serve as your unique individual extension is your DID. Numbers are essentially “rented” from providers. CLECs such as Bandwidth.com have the power to sell phone numbers and port them, but they don’t necessarily own them. 

2600hz
Now to actually explain what the 2600hz reference comes from. No, we didn’t name our company with four syllables just to give people a hard time when pronouncing our name, or to be the first alphabetized company at conferences. During the prime time of telephone switching and ‘hacking,’ people found that they could whistle the 2600hz frequency to trick the telephone operators in thinking that they had hung up a call when they were still on the line. They could then begin to make phone calls all around the globe without paying an extra dime. Since we are an open source platform, and we always promise to be one, we love the idea that people can start opening up the black box of telecom and start making calls freely with more flexibility and freedom than ever before. 

We have a little secret for you. Our team members have previously worked at places which carried millions of calls for Skype, Google Voice, Twilio and more. We’re all too familiar with poorly written APIs, slow number searches, DTMF problems, call quality issues, billing headaches and more. And based on our experience working at these firms, we’re committed to avoiding least cost routing, dialer and grey routes.

Based on Monday’s ruling by the FCC, we think this matters a lot. (Official document and details here.) We’ll explain why, but first, some background.

What is Least Cost Routing?
Least cost routing is a simple idea: sign-up for phone service from two or more phone companies and let your phone system decide which phone company it uses for every call, based on the lowest price offering. For example, if phone company A has great rates from 9am-5pm and phone company B has great rates at all other times, well, simple enough – let your phone system use the cheapest company based on the current time.

At least, that’s the idea.

Least cost routing (LCR) has evolved from simple strategies like the ones listed above to a massive market of buying, selling and trading minutes across a virtual black market. People input “rate decks” (or large lists of areas you can call which each have a different cost to call) into a computer from hundreds of different phone companies and shuttle around your calls based on whoever can honor the cheapest rate at that exact moment, for whatever reason.

The number of strategies is as complicated as the impact those strategies have. We’re only going to scratch the surface in this article of what happens in this market by talking about two items – rate blending and grey routes.

Blending Rates

Some companies lower their rates by “blending” rates together. For example, let’s say a phone company named “Super Bell” can deliver calls to Iowa at a cost of $0.03/minute and calls to California at $0.01/minute. The average rate per minute is $0.02 if traffic was even, and that sounds like a pretty good rate to most consumers. Super Bell markets this as a “flat-rate” in hopes that more people will call California then Iowa. (which makes sense since California simply has a lot more people.) If that works out, they’ll make money, since most calls will cost less then they will charge you. But you, as the consumer, are always guaranteed $0.02, which is nice and simple to understand and sounds really cheap!

But, wait! Along comes Mega Call. This phone company promises rates that can’t be beat. How do they do this? They set up a fancy computer program that takes the really expensive phone calls that their customers are making to Iowa (which would cost them $0.03/minute normally) and they start sending all those calls to Super Bell who advertises the $0.02/minute blended rate. Mega Call does NOT send Super Bell the less-expensive traffic to California and instead keeps it for themselves, paying only $0.01/minute for those calls. So, now Mega Call can market their service at $0.02 (the same as Super Bell) but will never have a single call that loses them money because all their expensive calls went to the other guy! They’ve got themselves a guaranteed win. 

Make sense?

Here’s the problem. This has all become very computerized with many, many companies participating in this behavior. The net effect is that some phone calls are literally bought and resold five times before the call is finally connected, so that people can provide you a cheap rate. In the process of doing so, sometimes calls get dropped, equipment fails or is incompatible with other equipment or call quality simply degrades. Some calls literally get bounced all the way from New York to California and back again even though you’re calling someone who’s just one state away. This inevitably impacts your call quality.

What’s a Grey Route?
In the above “blended rate” example we talked about how some companies gamble on flat-rate pricing with the expectation that the majority of calls will be to inexpensive areas and only a few calls to more expensive areas. But what happens when they’re wrong and they get mostly expensive calls instead of cheap ones?

Enter the Grey Route. The grey route is, literally, a sub-par phone line or phone company who is intentionally selling phone service in areas that should be expensive but is cutting corners to be able to provide the service for less. An example of a grey route, in it’s simplest form, might be someone buying 50 phone lines that were on special from the phone company for 6 months – and putting those phone lines in their garage. Then they buy an internet connection and funnel calls from the Internet to those cheap phone lines all day long.

Well, what’s wrong with that? 
First of all, the folks running the grey route don’t really care about the quality of the calls. They’re most likely using a poor quality internet connection, poor quality equipment and aren’t interested in debugging or fixing problems with their setup (like in the photo pictured above) – as long as they can keep you on the line long enough to bill the other party. How do they achieve that? They pitch the route to the phone company who’s losing money on expensive phone calls and falsely promise them great quality.

Remember that whole housing market scandal where people took poor-quality loans and bundled them with high-quality loans and marked the whole package as a “safe bet”? Well, that’s not the only industry playing this game.

In essence, the theory goes that if only 5% of your calls go over a “grey route” then phone companies can save literally millions of dollars and most customers will “tolerate” the poor quality because it only occurs on such a small number of calls. Unfortunately, the side effects of such behavior range from broken Caller ID and touchtone transmission to audio quality cut-outs and generally poor sounding calls.

What Does All This Mean? How Bad Has This Gotten?
Everything we just talked about comes down to a simple idea: as people are busy trading minutes to lower prices of making a call, they’re forgotten that there’s a human on the line who wants to make a call – and that’s you. A real, live, human being who just wants to call a loved one – or an emergency police department call. A call that’s important.

The problem of trading minutes resulting in poor quality phone calls has gotten so severe that the FCC has stepped in and, as of Monday, is threatening fines of up to $1.5 million for the worst offenders. While the FCC is trying to make a statement, it should be understood that such a large fine would only be utilized on companies like Verizon, AT&T, Vonage, Skype or others of that size who could actually pay that amount. In other words, the FCC is saying in no uncertain terms that it knows this problem goes “all the way to the top.”

Why We’re Proud of Our Services
Yup, pitch here. But for good reason. We spend a lot of time researching our carriers. A LOT of time. We don’t pretend to be a phone company who owns equipment – we resell other people’s minutes but we use our knowledge of providing quality service to decide who to use. We don’t tolerate grey routes and we insist on working with providers who have direct access to the folks we’re asking them to call.

2600hz is proud to have researched and selected carriers who:

• Only provide direct routes – that means they own the termination equipment, they don’t pass it to someone else
• Have invested in their infrastructure with equipment we trust (yes, we checked!)
• Have a proven track record for redundancy and reliability
• Have NOC and tech folks we can contact to trace individual call failures

2600hz is one of the only companies in the industry to openly promise direct routes. We are also one of the only companies that doesn’t touch the audio in your phone call with our wholesale services – we don’t even proxy it. We basically handle billing, contracts, E911, DID purchasing and inventory, porting and general management — and we pass the volume discounts to you.

In Conclusion
There is no such thing as a phone company that has equipment everywhere. No such company exists since the Ma Bell monopoly was divested. All companies – including Verizon, AT&T, Sprint, etc. all must work to integrate with each other and are allowed to service only certain areas of the country. This is a great thing as it encourages competition.

The question now becomes whether we can be competitive within the guidelines of minutes trading without compromising call quality. It is this quest which 2600hz is after – improving the VoIP experience for all.

For this week’s Tuesday Tip, we asked our community to share some of the most frequently used telecom terms you’d like to see defined and unpacked. So we will be spending this week and the next to unpack some of these and other commonly used telecom terms you may have heard us or others throw around. 

SIP
Short for Session Initiate Protocol, it’s the internationally recognized signaling protocol used in VoIP.  It’s the way that phones communicate and know how to speak to each other when connecting to make calls and to hang up calls. We’ve gone into this at much more length here and here.  

Trunks
The easiest way to think about trunks is to think of them as a ‘vanilla phone line’ or a pipe created for communication through which minutes and other features are funneled through. Similar to getting minutes for your cellphone in order to make and receive calls, trunking is basically the work of providing minutes/phone calls for a multitude of people through one set of lines. Very much like the trunk of a tree, the actual trunk is the same running across all audiences but branches reach different sets of people and their phones. 

Callflows

The way you want your calls to be routed throughout your PBX or enterprise office. You can create and set the order of how a call will ‘flow’ when someone calls your office number. Look at this one, for example. I’ve set up a call flow for  Peter’s IT Shop. When someone calls his main business number, the caller will reach a menu that allows him to reach either Peter, Sam, and dial into a conference bridge if necessary. 

Time Of Day
Time of Day is a feature that allows you to adjust the phone settings of your business phone, by the hour. So you can change and specify how you would like your phones to ring at what time of the day. An office, for example might want to have their phones go straight to voicemail from 6pm to 8am, but ring the normal call flow during business hours. 

Global Resources
Global Resources allow you to route calls to the phone network through pre-configured service providers meaning any upstream carrier (Bandwidth.com, Verizon Wholesale, AT&T Wholesale, Level 3, iCall, etc.) that you might bring to route your calls to the outside world. 

Cluster Manager
This is our server manager that allows you to automatically add and delete servers. Logs will allow you to view and manage all your servers based on location, Server IP and etc. We’ve gone into more detail about this here. 

DTMF
DTMF stands for Dual-Tone-Multi-Frequency. It seems like a mouthful, but in actuality it’s pretty simple. This takes us way back to the first time phone keypads were set in the order that they are now.  If you look at the traditional phone keypad, you could probably close your eyes and know which numbers are where and that’s because keypads are pretty much all the same. Beyond being a matter of comfort and habit, the reason for this is because the number pressed is specified by the row and column that number is positioned in. Since each number has a specific location and ‘tone’ the combination of these tones lets the hardware know how to switch the call properly. 

Fun fact: DTMF was created in order to prevent hackers from using their voice to mimic the same tone which is how the 2600hz tone was created. That’s why there are now two tones (one low frequency and one high frequency) to allow the phone company know which number to dial. 

ATA
ATA stands for Analog Telephone Adapter. These devices cost roughly around $30 and when plugged into your phone, will turn any phone around into a SIP device. So no matter what kind of phones you have, all it takes is a $30 device to turn your phones into a VoIP phone. 

We’ll continue to cover more terms in next week’s Tuesday Tip including PSTN, Cloud, 2600hz, and DIDS. 

For this week’s Hacker Stories we’re shaking things up a bit. With Facebook’s recent S-1  this past week, and Zuckerberg’s lengthy discussion about his company’s  ‘Hacker Culture’, we thought it would be fun to share our own ‘hacker stories’ and the ideas that we’re cooking up in our own kitchen every now and then. 

Last week I was sitting in a roundtable discussion on customer service when I learned that Eventbrite is still mainly using phone support for many of their customers. They explained that their customers often call hours or even minutes before their event starts and prefer the immediacy in getting their questions answered and problems solved hours over the phone while speaking to a voice rather than a person behind the screen. On the other hand, chat support solves the problem of waiting in a never-ending queue of waiting, since the chat assistant can be ready to actually talk to you whenever the chat conversation starts. Then the creative juices started flowing. (Though I will also admit I was motivated by a general greed for having the best of both worlds.) Couldn’t there be some way of combining the two? What if you could get the same human connection of phone support without having to wait on hold without the terrible music and unnecessary ads on repeat while you’re waiting? 

With our platform, we imagine one can create a support call center where callers can call in, leave their message and number to add themselves to a queue. Then the customer service representative on the other side will reply to the customer’s call through their CRM that would give them some background context and information to the customer about their latest event. This not only frees up the customer from being tied to their phone on hold, but also gives the customer service rep plenty of information to the caller so that the call can start much quickly without the customer explaining all their history and background.

Providing customers with a positive customer service experience is one of the best ways a company can set themselves apart while instilling both trust and love for their brands. (Great example of how another event organizing company is still holding true to this belief.) We’ve built a platform that we’ve customized to be as flexible as can be for situations and purposes such as these. We hope you’ll take advantage. 

At 2600hz we like to say that we’ve “re-written the telephony stack.” Which either elicits looks of belief and wonder or hardcore skepticism. Which is fair. So we’re going to be temporarily replacing our usual Wednesday Bug series with a short series on the architecture stack we’ve designed to improve telecom like it’s never been done before. The seven components are: 

• Session Border Controllers (for SIP)
• Load Balancers (for SIP)
• Media Servers
• SMS / MMS / SMPP
• Dialplan /  Call Processing
• Messaging Buses
• Databases

Today, we’ll talk about two of those components and our experiences using them. These components are OpenSIPs—our load balancer, and FreeSWITCH—our media server. 

Why We Selected These Components
OpenSIPs has a long history of being an extremely stable, flexible SIP proxy, allowing you to set up a load-balanced, redundant setup for many millions of calls per day. One of its greatest strengths is its rock-solid reliable performance and its ability to do reasonably complex tasks in a short amount of code. But OpenSIPs can’t natively handle audio and frankly, wasn’t really designed to.

FreeSWITCH has a long history of being an extremely stable and flexible media switch,  allowing you to handle many thousands of simultaneous phone calls on a minimal amount of hardware without degrading call quality. In addition, FreeSWITCH has built-in functionality to support integration with OpenSIPs which is crucial as it would otherwise be hard to work with.

How OpenSIPs and FreeSWITCH Are Different
At the fundamental level, OpenSIPs and FreeSWITCH serve two different purposes. OpenSIPs was originally designed to be a scripting engine and a proxy for SIP packets. In English, think of it as the “traffic cop” with the neon vest that stands in a busy intersection. You can program it to tell it how to direct traffic in different ways, based on volume, quality or location or other factors.

So if we say OpenSIPs is the load balancer that directs traffic (call flow) along the road which is the Internet…where does all that traffic get directed towards? You guessed it. FreeSWITCH. 

FreeSWITCH is a media server. Think of a media server as being similar to a language translator between two people who speak two completely different languages. In its simplest form, a media server takes audio from one person, processes it, and passes it on to another person. It also provides translation services, meaning if one person “talks” another language it can convert that language on the fly to something the other party can understand.

Technically speaking, there are major differences in how you would architect a traffic cop versus a translator. The analogies to real life work well, so lets continue with those.

In the OpenSIPs as “traffic cop” analogy, think for a minute about the length of time a traffic cop is concerned about the traffic flowing around it. A traffic cop directs cars down a street, but once the cars pass by, the cop no longer cares who they were, where those cars go,  or what direction they continue to travel in. In this regard, a traffic cop may see millions of cars pass them per day but is only ever concerned with the few cars it is currently directing at any given time.

In the FreeSWITCH as “translator” analogy, lets contrast the length of time a translator must focus on a conversation between two people. (For the purposes of this analogy, we will represent a FreeSWITCH box as a building filled with thousands of translators.) A translator must focus on only one conversation at a time while being able to understand and speak the language of both sides. The translator can’t leave in the middle of a conversation easily to be replaced with another translator – they must stay for the duration of the entire conversation.

Comparing OpenSIPs and FreeSWITCH For Capacity (and why you shouldn’t)
I’ve seen lots of people try to compare the capacity of OpenSIPs to that of FreeSWITCH. This doesn’t make much sense. Looking back at the above analogies we can see the fundamental differences between OpenSIPs and FreeSWITCH.  In technical terms, this translates into different ways to analyze capacity based on work done by threads.

Both OpenSIPs and FreeSWITCH utilize threads in order to process calls. The difference lies in what the threads are doing.

For OpenSIPs you have a Linux thread running which is the equivalent to a traffic cop. A single thread might pass 10,000 calls per hour from point A to point B, processing a light-weight instruction script (“is this person authorized? OK, let them through!”) for each call. The real concern around capacity for OpenSIPs is how many simultaneous requests OpenSIPs will get at any one point in time. Think of this as the difference between a four-way intersection and an eight-way intersection. You might need only one traffic cop at a four-way intersection but need an extra cop at an eight-way intersection simply because of the extra load.

For FreeSWITCH you have a Linux thread running which is the equivalent of one translator. A single thread will only ever process one call, for the entire duration of that call.

So, when planning capacity, you need to ask yourself:
OpenSIPs – How many simultaneous people are going to register a phone, make a call or hangup on a call at the same exact time? Divide that number by how many calls each thread can process and you know how many threads you need.

FreeSWITCH – How many simultaneous people are going to be on the phone at any given time? That is how many threads you need.

Now that we understand the difference between OpenSIP and FreeSWITCH and their purposes and capacities, stay tuned for next Wednesday’s post on FreeSWITCH support for OpenSIPs (or any proxy.) 

(click to zoom into images) 

For this week’s Tuesday Tip we are excited to give you a sneak peek into our (BETA)  cluster manager. While we still have lots more planned for how we want this to ultimately look and operate, we’re excited to show you the first peek at how this tool will make deploying and maintaining your servers that much easier. 

Deploying your own server hasnt been the easiest cake walk on our platform so far. Servers were not visually visible, you couldn’t manage it without logs and it was difficult to monitor the various roles on your cluster the way you wanted to. Today, we’ve made it easier for you to manage your server clusters, whether you have one or ten. You will now have an easy one-stop portal to manage your server and adjust each one. While you’ve been able to add new servers on the fly with our previous cluster manager, this new redesign allows you to designate the role of each server as well. 

Again, it’s important to note that this is currently in BETA version.  In the next few weeks, you can look forward to clicking ‘Status’ to turn individual components of your cluster on and off when you need to handle maintenance. While there is currently a working version, we’ll also be polishing the log so you can have a record of every occurrence on each individual server. 

To take a look at your own cluster manager, log in to your account in our GUI. 

For a city thriving with freelancers, new entrepreneurs and small companies graduating from the living room floor but waiting to invest into a corporate office, NextSpace provides co-working office space that doesn’t suck. Members receive free legal advice sessions, weekly happy hours, zip car membership, and phone access just to name a few of their perks. But with over 165 regular members, they needed a system that would allow users to create custom caller IDs for those calling from NextSpace phones that still wanted to appear they were calling from their designated business or cell phone number. The enterprise-grade flexibility they needed was exactly what 2600hz was built to be. 

In the world of Skype, email, and cell phones why would anyone still need to use an actual phone to handle business you wonder? “It’s mostly AT&T users because we get terrible reception here so handling important business calls over cell isn’t very reliable” said Lindsay Burstedt, Community Manager at NextSpace. 2600hz phones relieve users worried about poor cell service and shoddy audio quality. Members also get the free minutes bundled into their NextSpace membership. 

How Dynamic Caller ID Works
When making a call, the user will be prompted to enter their Caller ID number and the call will replace the actual caller ID to display the one typed in. The goal of offering this service was to make it easy for anyone to use without reading a manual or getting a tutorial. ”It was really intuitive. We just sent out an email to all our members about this feature and everyone started using it. It was pretty nifty that we didn’t have to add the 1 in front of the phone number before dialing too.” 

Transitioning to our system didn’t even cost extra effort to switch to VoIP phones, since a simple ATA converter like this one easily turned their regular analog phones turned into SIP devices with little effort and time. 

Hacker Stories is a weekly series highlighting the way that developers and customers are building and expanding the 2600hz platform. If you’re building something cool with the 2600hz platform or using it to do something cool, we want to talk to you! Email rachel@2600hz.com or tweet us at @2600hertz. 

A couple days ago I walked into a small office that was frustrated with their AT&T service and poor customer service (no real surprise there). When I asked if they had considered a VoIP/Cloud Telecom solution the conversation naturally turned into a “So..why is cloud telecom a better idea for my office and how can I save money with it?” discussion.

Since a lot of people have wondered about the benefits of having a cloud communication system, I’ve decided to compile them all together in one list. 

1) Fewer in-house experts needed to manage what you’ve got
There’s nothing worse than having an expensive new system that no one in your office knows how to manage. Now you can have the cutting-edge software and know how to configure it through an easy web interface that works on the intuition you already have. A simple drag and drop will allow you to control the way a call is made into your office and will be put into action in real time (no more waiting around for the one IT guy in the office to schedule it in). The same GUI will also set you up with a new voicemail box, call forwarding, and time of day features to set business hours for your office phone. And the best part? You can do this all from your desk and not in the scary basement where an intimidating box resides. 

2) Lower Total Cost of Ownership
Businesses want and need the latest features and systems to stay competitive in this economy. But it’s difficult to drop the big bucks right at the beginning. With cloud communications, there’s no need to pay for the expensive hardware at all. In fact we recently ran 400+ concurrent calls on 512MB servers which cost less than $200 total in hardware. While this is true for any cloud communication service, the 2600hz architecture allows you to assign commodity hardware such as 512 MB servers to handle less security-sensitive items such as call switching while using other more expensive ones to handle data storage, voicemails, etc. In terms of phones, you can use any SIP device to get started on our platform - expensive or cheap, choose the devices that match your budget. 

3) Scale Up or Down
Businesses change all the time and it becomes difficult to project the next five, ten years. So when you invest in any technology, you want to make sure you’re not locked in forever. Whether you need to open a new office tomorrow, or cut back three branches the next, cloud communications allow you to expand and retract as you need without getting locked into clunky hardware to keep up with you. 

4) Stay up and running, 24/7 Business Continuity
Phones are the core of many businesses, to the point where tons lose money and customers if their phones are not constantly up and running. (Lives can be on the line too.) We get that, so we’ve built an architecture designed to fail so that we could test its redundant reliability. You read that correctly, we’ve designed it knowing it can and will fail. That means that even if something should happen mid-call (a fire starts, an earthquake strikes, someone trips over a cord) and the database running your call suddenly goes down, your call will automatically fail over to another data center, without disrupting your call or business at all. Business proceeds, as usual. 

5) Get The Features You Need
Caller ID is the last feature that traditional telcos have offered, and we think that’s a bit 1990’s. We’ve got features like voicemail to email forwarding, music on hold, call forwarding between your desk phone, soft phone, home and cell phone, dial by name directory and hotdesking to name a few. They’re all designed with the business environment in mind, in hopes to increase business efficiency without making you work harder. 

6) Upgrade Without Breaking a Sweat or Paying Another Dime
Upgrading old systems used to mean buying a new box or system, similar to buying software updates on CDs (remember that?) Cloud communications is the equivalent to replacing the upgrade CD with a download update. Back-end upgrades happen unnoticeably in the background while new features and capabilities can be downloaded to the newest version at no additional cost.

7) Customized Functionality That’s Not Scary
All businesses are different—yours is nothing like the next one. Our customization features allow you create the system that you need for your business, with customized greetings, ring groups, music on hold and white-label software. All which can be configured in our web interface which lets you look under the hood of your system without scaring you off. When it comes to hardware customization, use any SIP device you’d like. So if that means getting the CloudTC Glass for executives and Yealink for everyone else, you’re free to create your own customized combination of phones. 

8) Mobility anywhere
Employee mobility is ever expanding as business takes place globally and more people are working on the go. So whether you’re working at your desk, in a separate conference room, or on the road, cloud communications allows for you to take your call, wherever you are. 

Just like no one buys CDs anymore to buy programs when they can download new software, more businesses are rapidly moving to cloud telecom. Consider if it’s time for your business to do the same. 

This is an ongoing part of our blog series. Read our first Hacker Story here. 

One of the biggest parts of the 2600hz mission is open accessibility in telecom. That’s why we’re completely open source, with complete access to our APIs. So we get a little bit more than excited when we see the way that developers and companies such as Stratafon use our Crossbar API to interface with the 2600hz Platform. 

You hate voicemail, right?  Voicemail is one of those double-edged swords. You need them, so you’ll never miss an important message, but it’s tedious to manage them all and you’re sick of old messages clogging up your inbox. Some are important and you’d like to keep them around, and others are the annoying ones you wish you never wasted the time to listen to. 

Stratafōn, the VoIP division of Teal Networks has created a portal that hopes to put an end to all your voicemail woes. Their voicemail management service will allow users to listen, download and delete voicemails as well as return calls from within the browser via click-to-call. It will also allow users to manage PIN numbers, e-mail, call forwarding and view all call records from your extension. No more staying on the phone listening to automated menus for minutes on end, just to edit your voicemail box settings. 

Greg Cronin and the rest of the team at Teal Networks have always desired to give clients a simple interface to manage their phones. “Unfortunately, every third party platform we’ve used until now has been difficult to integrate with or has not offered all the features our clients required.” The team ventured to develop their own system but ran into a number of issues ranging from troubleshooting, server failures and customer relations - problems serendipitously resolved upon discovering the 2600hz platform. “After a few phone conversations with Darren and some demos of the system and API capabilities, we were hooked. From there it’s just been a matter of learning how to interface with the Crossbar API  which the 2600hz team has been immensely helpful with.” 

The Crossbar API is a general purpose high-level API that allows any external third party to interface with the 2600hz Platform while incorporating access to permanent storage.  It’s an abstraction layer that exposes powerful APIs while hiding all the complexity. “As with any API, there was a bit of a learning curve but once we got moving with it, it’s been intuitive and easy to use. The team has provided great support enabling us to quickly develop a powerful interface.” 

If you’re a developer doing something cool in telecom with 2600hz, we want to talk to you! Email rachel@2600hz.com or reach us at @2600hertz.