Topic: SME8FA18V311-16 DAHDI Fix

[apmuthu]

Installed SME8Final + Asterisk v1.8 + SAIL v3.1.1.16 from the consolidated ISO at:
http://sailpbx.com/iso/SME8FA18V311-16.iso

Upgraded SAIL to latest available SAIL using:

Code: [Select]

cd /usr/src
wget http://sailpbx.com/sail/sail-3.1/sail-3.1.1-17.noarch.rpm
yum localinstall -y sail-3.1.1-17.noarch.rpm

Rebooted from GUI.

The current versions are:

Code: [Select]

# rpm -qa | grep sail
smesailenvast18-1.0.0-1
sail-3.1.1-17
smesailenv-1.0.0-30

# rpm -qa | grep asterisk
asterisk-sounds-core-en-alaw-1.4.21-1_centos5
asterisknow-version-2.0.0-4_centos5
asterisk18-addons-core-1.8.7.0-2_centos5
asterisk18-addons-mysql-1.8.7.0-2_centos5
asterisk18-dahdi-1.8.7.0-2_centos5
asterisk-sounds-moh-opsound-alaw-0.0-4_centos5
asterisk18-core-1.8.7.0-2_centos5
asterisk18-voicemail-1.8.7.0-2_centos5
asterisk18-doc-1.8.7.0-2_centos5
asterisk18-configs-1.8.7.0-2_centos5
asterisk18-1.8.7.0-2_centos5

# rpm -qa | grep kmod-dahdi
kmod-dahdi-linux-fwload-vpmadt032-2.6.1-1_centos5.2.6.18_308.13.1.el5
kmod-dahdi-linux-2.6.1-1_centos5.2.6.18_308.13.1.el5

# uname -a
Linux s01 2.6.18-308.13.1.el5 #1 SMP Tue Aug 21 17:10:06 EDT 2012 i686 i686 i386 GNU/Linux

The smesailenvast18-1.0.0-1 rpm is available only in the ISO and not in the repo.

yum update from console. This may risk setting the kmod-dahdi* rpms out of synch with the kernel unless the asterisk and dahdi are also updated from the digium and asterisk current and/or 1.8 repos.

Code: [Select]

# uname -a
Linux s01 2.6.18-308.24.1.el5 #1 SMP Tue Dec 4 17:42:30 EST 2012 i686 i686 i386 GNU/Linux

# cat /etc/redhat-release
SME Server release 8.0

# cat /etc/issue
AsteriskNOW 2.0.0

The IP address of your AsteriskNOW system is 192.168.1.100

# rpm -qa | grep sox
sox-12.18.1-1.el5_5.1

Create missing folders to avoid error messages:

Code: [Select]

mkdir -p /var/lib/asterisk/firmware/iax
chown -R asterisk:asterisk /var/lib/asterisk/firmware

DAHDI FXO will not work out of the box as there is an error in the dialplan generator that is corrected by inserting the following at Line 383 in /opt/sark/generator/extensions.php :

Code: [Select]

exten => s,n,Set(chan=\${CUT(chan,-,1)})
Note: This fix is also applicable to SAIL v4.0.0.16 and SAIL v4.0.0.26 deb - Insert at Line 386.
 
In the sail gui - http://SMESERVER.IP/sail - login with the SMEServer credentials and navigate to
End Points -> Trunks -> Create New Trunk -> Custom -> Next

Asterisk Peer : DAHDI1
Dial String Lead-in : DAHDI/fxo/1
Dial String Lead-out :
Routable : YES
Name : DAHDI1

Note capitalisations used in the above as they are faithfully used in context creation and usage in the generated dialplan.

Edit the newly created DAHDI1 Trunk and set the Open and Closed destinations before saving and committing the dialplan.

The System -> Globals -> Real Operator Extension is used as a destination of last resort - a sort of fall thru' target.

[SARK devs]

wow, lot of work gone into that post.
Could you show the lines either side of your modification please?

Kind Regards

S

[apmuthu]

Considering the wonderful work the SARK/SAIL team has done and gifted it to the community, the three days and nights spent understanding and fixing it pales in insignificance. Thankyou and keep up the good work.
 
The patch file contents for the SAIL v3.1.1.17 is:

Code: [Select]


--- orig_sail_31117/opt/sark/generator/extensions.php Mon Oct 08 02:43:27 2012
+++ new_sail_31117/opt/sark/generator/extensions.php Sat Jan 05 20:55:49 2013
@@ -380,6 +380,7 @@
 [from-pstn]    ; FPBX Compatibility
  include => mainmenu
  exten => s,1,Set(chan=\${CUT(CHANNEL,/,2)})                 
+ exten => s,n,Set(chan=\${CUT(chan,-,1)})
  exten => s,n,GoTo(mainmenu,DAHDI\${chan},1)
 
 HERE;


Also tested and works with yum updated SMEServer 8 (new kernel) and Asterisk v1.8.19.1-1 taken from the asterisk-1.8 repo instead of the original asterisk-current repo that is now lagging. The digium-1.8 repo is also slightly ahead of the digium-current repo but is grossly insufficient to be used standalone.

[SARK devs]

Thanks apmutha

We'll look at fetching this into the next release.   Our analogue testing on i386 is not extensive because we have so few analogue deployments other than on the PIKA warp, which has its own bespoke analogue cards and logic.  Almost everything else we do is BRI/PRI or pure VoIP.

Best

S 

[apmuthu]

That would be nice. Actually most home users would want to test out SARK/SAIL on generic i386 pcs or possibly virtual machines. Even sysadmins intending to go in for asterisk would be such potential users. If their first impression is of a successful install / clear and familiar nomenclature, then their continued participation in the community will most likely follow besides advising their clients to go in for the eco-green SARK hardware units as they would have become comfortable with it's interface.

If there is a GitHub repo for SAIL it would be nice. Also the source code for the sark agi binary would amount to trusted usage.

There are a few scripts, css and images that are not used in each SAIL distribution but possibly retained for a common base rpm. Some have hardcoded email ids in them and others have their title listed as SMEServer8beta5 like:

/opt/sark/scripts/daycall - not used - Line 101 - ReplyTo email address is set as admin@selintra.com - emails out daily calls list

/opt/sark/scripts/srkchkip.pl -> not used - Lines 44-45 send mail from admin@aelintra.com to jeffstokoe@gmail.com when IP changes - possibly used for debugging

sark-common/
  userpassword_head.tmpl,
  head.tmpl,
  noframes_head.tmpl
- title: SARK Server 8.0beta5 => SARK Server 8.0

A few tool tips are missing in the default sqlite3 db and some need to be more explanatory like ACD->Greetings when no greetings exist - The following can be added to the help message - "Dial *60*nnnn to record your greeting where 'nnnn' is the greeting number suffixed to the requisite prefix of 'usergreeting'".

Write access to SAIL wiki requested so as to update many missing images and keeping the generic i386 pages up2date.

[SARK devs]

Thanks for this

If you'd like update access to the wiki then email me at admin@aelintra.com and I'll send you login creds.

Kind Regards

S

[apmuthu]

Sent Request email. Awaiting access.

[apmuthu]

Thanks - received access email with username as "apmutha" instead of "apmuthu".

SAIL v3.1.1-20 rpm and sail-4.0.0-34 rpm have been released. Any public changelog / repo for diffs?

The fixes suggested in this post have not been incorporated as yet.

In SAIL v3.1.1-20, in the file
/opt/sark/www/provisioning/snom/snom.php, the entry for the device Snom 821 is missing in the $types array.

Diff Patch from v3.1.1-17 to 3.1.1-20 is available here.

A manual changelog listing from v3.1.1-17 to v3.1.1-20 is as follows:

Code: [Select]

/etc/cron.d/sark
Enforcer perl script line enabled to cut off calls with no credit
Removed line calling missing srkstst.sh SARK Start Stop script

/opt/sark/amacs
r1620 : retained only db sql entries. Inducted Snom 710, 720, 760, 821 Device entries
r1673 : Inserted Device and htm entries for those taken out of r1620 and the new devices
r1751 : DB SQL entries for SIP / Provisioning MACs

/opt/sark/generator/srktimer.php
Cluster Parameter subscript fix

/opt/sark/perl/modules/sark/SarkSubs.pm
Enabled AutoCommit for SQLite db while handle is created, Unset db handle when done
SQLite sql execution now made into a prepare and execute command set

/opt/sark/www/cgi-bin/sarkinternal.pl
predominantly removed commented out lines
regressed SarkSubs::SQLiteGetHashKey to SarkSubs::SQLiteGetHash
placed '$hashScrnVars{pkey}' in quotes
removed print STDERR "$cfwd is $hashScrnVars{$cfwd}\n";
removed final print STDERR "PKEY IS $pkey";

/opt/sark/www/provisioning/snom/snom.php
$types now includes the new devices Snom 710, 720 and 760 but leaves out Snom 821 ????

A number of Thumbs.db seem to continue to remain in the rpm.
These are Windows leftovers in the /opt/sark/www/sark-common subfolders



[apmuthu]

Upgrade SAIL with:

Code: [Select]

cd /usr/src
wget http://sailpbx.com/sail/sail-3.1/sail-3.1.1-20.noarch.rpm
rpm -Uvh sail-3.1.1-20.noarch.rpm

The /opt/sark/db/sark.db has some database inconsistencies from past installs and current unfixed bugs:
1. There may not be any primary key declaration in the vendorxref table if the original install was v3.1.1-16 from the ISO. The only manual way to create a primary key in SQLite is to rename the table and create a new one with the primary key and import the data into it and drop the renamed table. Otherwise use a tool like SQLite Administrator.
2. In the Device table there will be 2 entries for Aastra VXT with the pkey as Aastra VXT and AastraVXT. Delete the latter with:

Code: [Select]

DELETE FROM Device WHERE pkey = 'AastraVXT'3. In the Device table, there is a typo in the desc field in one record corrected with:

Code: [Select]

UPDATE Device SET desc = 'Yealink descriptor' WHERE pkey = 'y000000000000.cfg';4. There are many erroneous entries in the undolog table all having the sqlstmt field as BEGINTRANS all of which can be deleted with:

Code: [Select]

DELETE FROM undolog WHERE sqlstmt='BEGINTRANS'5. Enhance the message when there is no greeting available with:

Code: [Select]

UPDATE tt_help_code SET displayname='There are no greetings recorded on the system. Dial <p>*60*nnnn</p> to create a new greeting, where nnnn is a 4 digit number.' WHERE pkey='Sarkgreetingmsg1';

[SARK devs]

sincere apologies for mistyping your handle apmuthu.

The work you have done is very impressive.  I suspect you know the product considerably better than we do!  I've already updated the Snom-821 array reference in the 3.1.1 source.  Thank you.

With regard to your request for diffs, we don't really keep them although we can get 'em from svn if we need to. We do keep a brief changelog in the rpm/deb.

Main change in -33/-34 is that we spawn an orphan copy of the database for the exclusive used of the AGI.   You probably already know that the AGI never updates the database, however, we have seen some lock situations in very busy systems. I was looking at a system earlier in the week that was happily sustaining 100+ active calls (200+ channels) but it showed a couple of lock incidents when we were issuing commits from the browser.  There is also a bug fix for XML source handling in -34.  This is to do with display and retrieve of XML in provisioning streams, mainly for Snom and Cisco/Linksys phones.

Your other fixes and observations will go in shortly.  Thanks again for this very valuable contribution.

Kind Regards

S

[apmuthu]

Keep up the good work guys!

Just an extra eagle's eye for you.

I just tried TortoiseGIT on my local machine to spit the diffs and study the code evolution. Analysing the install from 3.1.1-16 in the ISO thru to -17 and now -20 (manual) it is indeed a nightmare if we have to migrate across many builds.

The database changes will have to go into the startDBSQL file as well. There seems to be quite a lot of duplication in DBSQL as well and the lineIOdump and finally all in the startsark.db sqlite file

Which version will you all be focussing on - v3.1.1 or v4.0.0?
Is it going to live tightly with SME or as a standalone in any CentOS / Debian?

To make code management easier, best to have a common code base for all the SAIL code that do not need compilation of any sort. The OS interaction code can be a separate rpm:
SAIL_ver_core.rpm
and any one of
SAIL_SME_glue_ver.rpm
SAIL_CentOS_glue_ver.rpm
SAIL_Debian_glue_ver.rpm

for each Asterisk version!

Why not keep the SVN / GIT open for all to be atleast read-only?

[SARK devs]

Hi Apmuthu

There is redundancy in the package which should be removed but we've never gotten 'round to it.  For example, you mention DBSQL, lineIOdump and startDBSQL, none of which are used.  They really should be removed. 

The system uses a more or less, pristine database, which is called sarkstart.db.  During an install or upgrade, the installer looks to see if sark.db exists.  If it doesn't then it copies startsark.db to sark.db.  Next, whether install or upgrade, we run a script which applies all of the sql fragments in /opt/sark/amacs in name sequence. The name is made up of a letter plus the svn commit number, so it's effectively applied in ascending time order.  There is redundancy in the fragments but by and large it doesn't matter. The process relies on the natural entity integrity of the database for its idempotency.  In other words pretty much everything it does is additive so a re-run will simply cause a lot of harmless errors because the row(s) will already exist.   It's not pretty but it's not bad and in theory it allows upgrade over any jump in release although I can't categorically rule out any corner cases.
I keep visiting the idea of drawing a line under the process and starting again but that would mean an absolute barrier to upgrade which would demand some kind of migrate.  On balance I think it is OK and we've used it throughout V3/V4.

We have 2 releases which we currently focus on; 3.1.1 and 4.0.0.  3.1.1 is effectively stable.  We aren't actively adding features to it any more, we just bug fix.

SARK integration with SME was deliberately loosened, beginning with V3.  This hasn't anything to do with our commitment to SME but is down to the fact that we run SARK on other platforms and CPUs where SME cannot run.  We have versions running on PPC and ARM (5 & 6).  By and large the central code is identical across all platforms.  However, each platform has its own environment pack, which in the case of SME is called smesailenv.   We already have a version for Squeeze which we run on the S200 and Raspi (just for fun) but there are no plans to make a general Debian release at the moment.

We plan to move the source to GiT and make it generally available but we just haven't gotten 'round to it yet.

I promise I will visit all of the redundancies you've pointed out before too long.

Kind Regards and thanks again

S     

[apmuthu]

Thankyou so much for stating the files not used and the method of upgrading - it certainly saves me a lot of time trying to fathom the code.

If the said pristine db is pre-updated and made readily available in each sail release wouldn't that save time for new installs and restarts from scratch?

One note about the SQLite v3 database in use: When I export out the SQL statements from the database and import back into a fresh one created using the latest SQLite Administrator / RazorSQL and then import the SQLs back, I find that all blank fields are given null values. Do the scripts processing the SQLite 3 sb treat null and blank values one and the same - will the integers if used in arithmetic be properly wrapped / cast and comparisons behave as intended?

Aside: The size of an SQLite 2 db (400KB) is a third in size more than it'se SQLite 3 (300 KB) counterpart.

[apmuthu]

Further analysis of the database upgrade method proves that not all updates will take place.

All INSERTs must be converted to INSERT OR IGNORE, followed by an UPDATE statement.
Refer: INSERT on FAILED UPDATE equivalent in SQLite

Case sensitivity may also be an issue as all cases are used in the database and update scripts without distinction.

A few records fail to make it to the tables - I'm now going thru' it with a fine tooth comb.

[SARK devs]

That would be a useful exercise.

Incidentally, the undolog table is redundant.  In the early V3 prototypes it allowed you to hit an undo button to reverse database changes but it proved to be cumbersome so we dropped it before the original release.  Table is still there but does nothing.

as far as I know, SQLite word-case is not important so, for example,  you can freely use INSERT or insert, however, it would look a lot neater if we used uppercase throughout (which, to our shame, we clearly don't).

From your previous post

If we were to use an updated db as you suggest, how would we best handle the customer data during an upgrade?

re NULL and blank.   To us they are the same.   Our programming practice is such that if a variable evaluates to true we will use it otherwise we will use the default.

Best

S

[apmuthu]

I am working on the upgrade issue for customers.

New installs must however be from the latest standpoint even if an always updated sarknew.db needs to be put in.

Take a decision about the earliest sail db you wish to support. Then make one consolidated update to bridge each major version - say 2.2 -> 2.3 -> 2.4 -> 2.6 -> 3 -> 3.1 -> 3.1.1 -> 4 (and 3.2 -> 4). Take the first known schema for each version and provide upgrades only to the latest version. Since there are only a few tables, a proper ERD can be published with notes on usage and way forward in development.

The following issues have been found so far.

The sarkstart.db Device table has entries for snom820.htm and snom870.htm - both have xml provisioning. Similarly named ones with htm provisioning is in r1673. Similarly Snom VXT with xml provisioning is present in the sarkstart.db and the same Device name with htm provisioning is in r1751.

Further duplication is observed in r1751 in the case of the Device Aastra VXT, besides the sarkstart.db itself has duplicates named differently - AastraVXT and Aastra VXT.

Many of the files in the amacs folder have dates that do not correspond with their revision numbers sequence. I take the date first and then use the revision numbers within them in ascending order.

I have prepared a consolidated sql file with comments prefixed with double dashes and removed double quotes where possible. I have retained the case sensitivity as per the table names (Carrier for carrier, Cluster for cluster, etc) if only for the sake of uniformity.

If we drop the undolog table it should not affect the SAIL install - am I correct?

Other than for the two big inserts in r1673 and r1751, I have managed to make all INSERTs into INSERT OR IGNORE followed by UPDATE statements. Quite a few ALTER TABLE ADD COLUMN statements are redundant and some being already done in the sarkstart.db itself!

When one SQL statement in a file fails, all others in the file fail to get executed. Hence the ALTER TABLE ADD COLUMN will cause gaps in such execution if and when they fail. I am going thru the sequence of checking for such breaks - 28th time now....

Why were the SPA-3000FXS and SPA-3102FXS deleted from the Device table in r1577 ?

I have retained them pending clarification.

The sarknew.db and the consolidated sql upgrade script for SAIL v3.1.1-20 are available here.