Subject: Re: [siprec] [dispatch] Lossless Recording in SIPREC

From: "Parthasarathi R" <[email-masked]>
Date: Thu, 6 Feb 2014 22:21:06 +0530

Hi Gerben,

I agree with Ram that lossless recording is within SIPREC requirement as mentioned in RFC 6341. Apart from REQ-5 in RFC 6341, some of the text related to High availability is mentioned in the note of this mail.

Also, the lossless recording shall be achieved by multiple means. One of the mechanism is explained in draft-ietf-avtext-rtp-duplication-04. IMO, let us discuss in SIPREC about the specific technical issue in lossless recording which is not possible to be achieved through the existing mechanism 

Including SIPREC mailing alias in this mail thread.

Thanks
Partha

Note:

 Use Case 10: High availability and continuous recording.

      Specific deployment scenarios present different requirements for
      system availability, error handling, etc., including the
      following:

      o  An SRS must always be available at call setup time.

      o  No loss of media recording can occur, including during failure
         of an SRS.

      o  The Communication Session must be terminated (or suitable
         notification given to parties) in the event of a recording
         failure.

REQ-021: The mechanism MUST NOT prevent high-availability
      deployments. 

> -----Original Message-----
> From: dispatch [mailto:[email-masked]] On Behalf Of Ram
> Mohan R (rmohanr)
> Sent: Thursday, February 06, 2014 9:50 PM
> To: Gerben Stam
> Cc: [email-masked]
> Subject: Re: [dispatch] Lossless Recording in SIPREC
> 
> Hi,
> 
> Please see inline
> 
> -----Original Message-----
> From: Gerben Stam <[email-masked]>
> Date: Thursday, 6 February 2014 8:40 pm
> To: Cisco Employee <[email-masked]>
> Cc: "[email-masked]" <[email-masked]>
> Subject: RE: [dispatch] Lossless Recording in SIPREC
> 
> >Hey Ram,
> >
> >I have had an off-line discussion with Andrew Hutton, chairman of the
> >SIPREC group.
> >He suggested to post this on the IETF dispatch group as it is a good
> >topic to discuss in the team.
> 
> Ok. I was not aware of this.
> 
> >
> >What would I need to do to bend this to SIPREC WG instead of Dispatch?
> >
> >Regards,
> >
> >Gerben
> >
> >-----Original Message-----
> >From: Ram Mohan R (rmohanr) [mailto:[email-masked]]
> >Sent: donderdag 6 februari 2014 15:45
> >To: Gerben Stam
> >Cc: [email-masked]
> >Subject: Re: [dispatch] Lossless Recording in SIPREC
> >
> >Hi,
> >
> >Is there a reason why you want to have this discussion in dispatch and
> >not in SIPREC WG which is meant to discuss the issues around SIP
> >recording ?
> >If you don¹t have any specific reason to do it here you may want to
> start
> >this discussion in SIPREC WG.
> >
> >I do have some comments on this topic but I think SIPREC WG is the
> right
> >place to have those discussions.
> >
> >
> >Regards,
> >Ram
> >
> >From:  Gerben Stam <[email-masked]>
> >Date:  Thursday, 6 February 2014 3:58 pm
> >To:  Gerben Stam <[email-masked]>
> >Cc:  "[email-masked]" <[email-masked]>
> >Subject:  [dispatch] Lossless Recording in SIPREC
> >
> >
> >Dear DISPATCH group,
> >
> >SIPREC is a great initiative to standardize the interface for
> capturing
> >Communication Sessions.
> >Session Recording is becoming more and more critical due to Compliance
> >and regulatory changes over the last years.
> >
> >The compliance market is requesting more than capture only these days
> >Confirmation is needed to acknowledge the entire Communication Session
> >was captured correctly.
> >RTCP Reports (rfc3611) will help to confirm complete handover of RTP
> >content.
> >
> >
> >
> >New requirement is Œlossless¹ Session Capturing.
> 
> Why is this a new requirement ? RFC 6341 (SIPREC requirements) already
> has
> a requirement for Lossless recording. See REQ 005.
> 
> 
> >Lossless indicating handover of content (RTP) is acknowledged by the
> >receiver AND in case of receiving issues, the sender will resend.
> >Reasons for loss may be UDP packet loss or receiver failing(over) and
> >temporary not able to accept content.
> >Current approach to address this Œlossless¹ requirement is using 2
> >independent parallel receivers.
> 
> There may be other approaches well. For example an SRC may buffer for a
> small duration to take care of the loss. Two parallel receivers still
> does
> not guarantee that recording is loseless as you can have UDP packet
> loss
> on both paths.
> 
> >
> >This requires the sender to send 2 individual streams, in fact 2
> >independent SIPREC sessions.
> >
> >Not sure if this is covered currently as supported SIPREC deployment.
> 
> 
> Current SIPREC does not have any such constraints. Depending on your
> implementation model you can have the same CS recorded by multiple SRC
> or
> have same SRC do multiple recordings of same CS.
> 
> Regards,
> Ram
> 
> > We do see implementations based on SIPREC supporting it.
> >Alternative is quick failover at receiver in case of failure but as
> >sender will send only once, this may lead to loss during failover.
> >
> >I am not aware of any rfc covering lossless content handover, but
> there
> >may be standards covering this already.
> >
> >
> >Looking forward to discussion on the mailing list. I may be at the
> London
> >event.
> >
> >Regards,
> >
> >Gerben Stam,
> >NICE Systems
> >
> >
> 
> _______________________________________________
> dispatch mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dispatch

---

From: Mary Barnes <[email-masked]>
Date: Thu, 6 Feb 2014 11:12:02 -0600

Please don't cross post as that makes it a nightmare for people who like to
put emails for various WGs in separate folders.  The IETF email system is
setup to avoid duplicates and also people are not subscribed always to both
lists and that results in messages in moderator queues.

It is a good idea however to post a note to the SIPREC mailing list that
this discussion is happening on the DISPATCH WG mailing list - including a
link to the discussion thread.

Thanks,
Mary.

On Thu, Feb 6, 2014 at 10:51 AM, Parthasarathi R <[email-masked]
> wrote:

> Hi Gerben,
>
> I agree with Ram that lossless recording is within SIPREC requirement as
> mentioned in RFC 6341. Apart from REQ-5 in RFC 6341, some of the text
> related to High availability is mentioned in the note of this mail.
>
> Also, the lossless recording shall be achieved by multiple means. One of
> the mechanism is explained in draft-ietf-avtext-rtp-duplication-04. IMO,
> let us discuss in SIPREC about the specific technical issue in lossless
> recording which is not possible to be achieved through the existing
> mechanism
>
> > >New requirement is OElossless¹ Session Capturing.
> >
> > >Current approach to address this OElossless¹ requirement is using 2
> siprec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/siprec
>

---

From: Gerben Stam <[email-masked]>
Date: Fri, 7 Feb 2014 09:04:57 +0000

Hi Partha and Ram,

Thanks for the feedback on Lossless, I will check the rfc's you indicated and get back on potential gaps in lossless recording requirements.
So far I have never seen lossless implementation combined with SIPREC apart from HA (options ping).

Be aware that lossless is not only to cover packet loss in UDP transmission (like FEC is additional parity blocks to cover packet loss).
Also the fact the SRS may not be ready to receive would need to trigger the SRC to buffer audio.
Again I will check the suggested existing standards .

Regards,

Gerben

-----Original Message-----
From: dispatch [mailto:[email-masked]] On Behalf Of Parthasarathi R
Sent: donderdag 6 februari 2014 17:51
To: 'Ram Mohan R (rmohanr)'; Gerben Stam
Cc: [email-masked]; [email-masked]
Subject: Re: [dispatch] Lossless Recording in SIPREC

Hi Gerben,

Thanks
Partha

Note:

> >Is there a reason why you want to have this discussion in dispatch 
> >and not in SIPREC WG which is meant to discuss the issues around SIP 
> >recording ?
> >Session Recording is becoming more and more critical due to 
> >Compliance and regulatory changes over the last years.
> >
> >Confirmation is needed to acknowledge the entire Communication 
> >Session was captured correctly.
> has a requirement for Lossless recording. See REQ 005.
> 
> 
> There may be other approaches well. For example an SRC may buffer for 
> a small duration to take care of the loss. Two parallel receivers 
> still does not guarantee that recording is loseless as you can have 
> UDP packet loss on both paths.
> 
> >
> or have same SRC do multiple recordings of same CS.
> 
> Regards,
> Ram
>

---

From: Gerben Stam <[email-masked]>
Date: Fri, 7 Feb 2014 13:56:19 +0000

Hey guys,

I have had a look at Lossless statements in SIPREC now (Uses Case 10 and REQ-021).
Control start of RTP the moment we state a=sendonly in the SDP.
This is to cover RS start delay due to late acceptance by SRS, buffering on the SRC. It does no cover mid RS loss of RTP.

draft-ietf-avtext-rtp-duplication-04
Explains dual RTP streams send by SRS with own SSRC/SRTP keying and even RTCP. Using a=ssrc-group:DUP 1000 1010.
Streams can take separate network routes to cover UDP packet loss in the network.
At the SRS side we need to create single recording from the combined streams and fill potential RTP gaps by the other RTP stream.
This is to cover UDP packet loss or even stream loss in the network.

Issue not addressed still is what to do if SRS goes down mid SR and needs to failover (HA).
Before the outage is detected by SRC, it would have send RTP for the RS, which was not received by SRS.
RTCP reports may notify the loss at SRC (at least if SRC sends receiver reports to SRS).
Will RTCP also support re-sending a certain RTP packet if the SRS did not receive it? 

This may look like a trivial issue as for each new CS a new SIP invite is send, which is true for IPT (Phone) environments.
In trading recording however, a CS lasts from the time the Trader log's on till he log's off. So all day.
Within this very long SIP session, multiple RS are triggered each with its Meta. 
SSRC may stay but RTP has VAD (activity detection, no RTP send if no RS)
With current strict regulation, any loss of recording needs to be reported. RTCP helps at SRC.
Still re requirement by the industry is moving towards lossless. Mainly acknowledge RTP received and ability to re-send even mid RS by SRC.

draft-ietf-avtext-rtp-duplication-04 may help but that will require to capture duplicate RTP streams on individual RTP loggers.
If one RTP logger would fail, CS continues on other logger.
SRC would need to re-invite an update the RTP destination for the failing RTP Logger to re-direct the stream to another node.
That would cover mid SR RTP Logger outage, but will raise complexity at the RSC side. 
Still an option tough we may advice to create 'lossless'.
draft-ietf-avtext-rtp-duplication-04 is more tailored to cover dual RTP routes (and dual NIC) between singe SRC and SRS.

What do you think?

Gerben

-----Original Message-----
From: Gerben Stam 
Sent: vrijdag 7 februari 2014 10:05
To: 'Parthasarathi R'; 'Ram Mohan R (rmohanr)'
Cc: [email-masked]
Subject: RE: [dispatch] Lossless Recording in SIPREC

Hi Partha and Ram,

Regards,

Gerben

-----Original Message-----
From: dispatch [mailto:[email-masked]] On Behalf Of Parthasarathi R
Sent: donderdag 6 februari 2014 17:51
To: 'Ram Mohan R (rmohanr)'; Gerben Stam
Cc: [email-masked]; [email-masked]
Subject: Re: [dispatch] Lossless Recording in SIPREC

Hi Gerben,

Thanks
Partha

Note:

---

From: "Dave Higton" <[email-masked]>
Date: Fri, 7 Feb 2014 14:04:36 +0000

The tried and tested method to prevent packet loss is to use TCP, isn't it?

Dave

-----Original Message-----
From: siprec [mailto:[email-masked]] On Behalf Of Gerben Stam
Sent: 07 February 2014 13:56
To: Parthasarathi R; 'Ram Mohan R (rmohanr)'
Cc: Gerben Stam; [email-masked]
Subject: Re: [siprec] [dispatch] Lossless Recording in SIPREC

Hey guys,

What do you think?

Gerben

-----Original Message-----
From: Gerben Stam 
Sent: vrijdag 7 februari 2014 10:05
To: 'Parthasarathi R'; 'Ram Mohan R (rmohanr)'
Cc: [email-masked]
Subject: RE: [dispatch] Lossless Recording in SIPREC

Hi Partha and Ram,

Regards,

Gerben

-----Original Message-----
From: dispatch [mailto:[email-masked]] On Behalf Of Parthasarathi R
Sent: donderdag 6 februari 2014 17:51
To: 'Ram Mohan R (rmohanr)'; Gerben Stam
Cc: [email-masked]; [email-masked]
Subject: Re: [dispatch] Lossless Recording in SIPREC

Hi Gerben,

Thanks
Partha

Note:

NICE Systems UK Limited ("NICE") is registered in England under company number, 3403044. The registered office of NICE is at Tollbar Way, Hedge End, Southampton, Hampshire SO30 2ZP.

Confidentiality: This communication and any attachments are intended for the above-named persons only and may be confidential and/or legally privileged. Any opinions expressed in this communication are not necessarily those of NICE. If this communication has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender by e-mail immediately.

Monitoring: NICE may monitor incoming and outgoing e-mails.

Viruses: Although we have taken steps toward ensuring that this e-mail and attachments are free from any virus, we advise that in keeping with good computing practice the recipient should ensure they are actually virus free.

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 07 Feb 2014 11:29:23 -0500

(I'm replying on siprec because that is where I received the message I'm 
replying to. It is fuzzy to me whether this should be on siprec or 
dispatch.)

> Again I will check the suggested existing standards .

Note: one possible approach for an SRC that is trying to achieve this is:
- buffer all the media for a CS.
- when it is done, establish an RS, using RTP over TCP
- send all the media and metadata.

That should all be valid now, except maybe use of RTP over TCP for the 
RS might not be covered. (I *think* it is allowed by the existing 
protocol draft, but that probably deserves some discussion.)

*If* you did that, and the RS failed before all was transmitted, you 
could always retry it. That *might* cause a problem at the SRS, getting 
duplicate info. So that might also require some sort of extension.

An intermediate approach would be like above, but starting the RS before 
the CS is complete, but retaining the local buffer for speed matching 
and in case it is necessary to restart the RS.

Of course it is a bigger burden on the SRC to buffer the CS. But it may 
be necessary if you really require a lossless solution.

	Thanks,
	Paul

---

From: "Ram Mohan R (rmohanr)" <[email-masked]>
Date: Tue, 11 Feb 2014 11:16:19 +0000

Please see inline

-----Original Message-----
From: Gerben Stam <[email-masked]>
Date: Friday, 7 February 2014 7:26 pm
To: Parthasarathi R <[email-masked]>, Cisco Employee
<[email-masked]>
Cc: "[email-masked]" <[email-masked]>, Gerben Stam <[email-masked]>
Subject: RE: [dispatch] Lossless Recording in SIPREC

>This is to cover RS start delay due to late acceptance by SRS, buffering
>on the SRC. It does no cover mid RS loss of RTP.
>
>At the SRS side we need to create single recording from the combined
>streams and fill potential RTP gaps by the other RTP stream.

There could be many ways you would solve it. Some of them are:
SRC detects that SRS is down using whatever means it knows of (OPTIONS
ping or some thing else) and immediately switches over to another SRS.
SRC detects SRS is down and starts buffering the media.
SRC may have more than one recording of the same CS - redundant recording,
to different SRS. So even if one SRS goes down the SRC would continue to
record to the second one.

The current SIPREC does not stop you from having a implementation that
does any or all of the above.

Regards,
Ram

> 
>
>This may look like a trivial issue as for each new CS a new SIP invite is
>send, which is true for IPT (Phone) environments.
>Still re requirement by the industry is moving towards lossless. Mainly
>acknowledge RTP received and ability to re-send even mid RS by SRC.
>
>draft-ietf-avtext-rtp-duplication-04 may help but that will require to
>capture duplicate RTP streams on individual RTP loggers.
>SRC would need to re-invite an update the RTP destination for the failing
>RTP Logger to re-direct the stream to another node.
>draft-ietf-avtext-rtp-duplication-04 is more tailored to cover dual RTP
>routes (and dual NIC) between singe SRC and SRS.
>
>What do you think?
>
>Gerben
>
>
>
>
>
>Thanks for the feedback on Lossless, I will check the rfc's you indicated
>and get back on potential gaps in lossless recording requirements.
>Be aware that lossless is not only to cover packet loss in UDP
>transmission (like FEC is additional parity blocks to cover packet loss).