class: center, middle, inverse, title-slide # 2.5 — Short Run Profit Maximization ## ECON 306 • Microeconomic Analysis • Spring 2021 ### Ryan Safner
Assistant Professor of Economics
<EMAIL_ADDRESS>ryansafner/microS21
microS21.classes.ryansafner.com
--- class: inverse # Outline ### [Revenues](#3) ### [Profits](#16) ### [Comparative Statics](#26) ### [Calculating Profit](#30) ### [Short-Run Shut-Down Decisions](#42) ### [The Firm's Short-Run Supply Decision](#54) --- class: inverse, center, middle # Revenues --- # Revenues for Firms in *Competitive* Industries I .pull-left[  ] .pull-right[  ] --- # Revenues for Firms in *Competitive* Industries I .pull-left[  ] .pull-right[  ] -- - .blue[Demand for a firm's product] is **perfectly elastic** at the market price -- - Where did the supply curve come from? You'll see --- # Revenues for Firms in *Competitive* Industries II .pull-left[  ] .pull-right[ - .hi[Total Revenue] `\(R(q)=pq\)` ] --- # Average and Marginal Revenues - .hi[Average Revenue]: revenue per unit of output `$$AR(q)=\frac{R}{q}$$` - Is *always* equal to the price! Why? -- - .hi[Marginal Revenue]: change in revenues for each additional unit of output sold: `$$MR(q) = \frac{\Delta R(q)}{\Delta q} \approx \frac{R_2-R_1}{q_2-q_1}$$` - Calculus: first derivative of the revenues function - For a *competitive* firm, `\(MR(q) = p\)`, the price! --- # Average and Marginal Revenues: Example .bg-washed-green.b--dark-green.ba.bw2.br3.shadow-5.ph4.mt5[ .green[**Example**]: A firm sells bushels of wheat in a very competitive market. The current market price is $10/bushel. ] -- .pull-left[ For the 1st bushel sold: - What is the total revenue? - What is the average revenue? ] -- .pull-right[ For the 2nd bushel sold: - What is the total revenue? - What is the average revenue? - What is the marginal revenue? ] --- # Total Revenue, Example: Visualized .pull-left[ | `\(q\)` | `\(R(q)\)` | |----:|-------:| | `\(0\)` | `\(0\)` | | `\(1\)` | `\(10\)` | | `\(2\)` | `\(20\)` | | `\(3\)` | `\(30\)` | | `\(4\)` | `\(40\)` | | `\(5\)` | `\(50\)` | | `\(6\)` | `\(60\)` | | `\(7\)` | `\(70\)` | | `\(8\)` | `\(80\)` | | `\(9\)` | `\(90\)` | | `\(10\)` | `\(100\)` | ] .pull-right[  ] --- # Average and Marginal Revenue, Example: Visualized .pull-left[ | `\(q\)` | `\(R(q)\)` | `\(AR(q)\)` | `\(MR(q)\)` | |----:|-------:|--------:|--------:| | `\(0\)` | `\(0\)` | `\(-\)` | `\(-\)` | | `\(1\)` | `\(10\)` | `\(10\)` | `\(10\)` | | `\(2\)` | `\(20\)` | `\(10\)` | `\(10\)` | | `\(3\)` | `\(30\)` | `\(10\)` | `\(10\)` | | `\(4\)` | `\(40\)` | `\(10\)` | `\(10\)` | | `\(5\)` | `\(50\)` | `\(10\)` | `\(10\)` | | `\(6\)` | `\(60\)` | `\(10\)` | `\(10\)` | | `\(7\)` | `\(70\)` | `\(10\)` | `\(10\)` | | `\(8\)` | `\(80\)` | `\(10\)` | `\(10\)` | | `\(9\)` | `\(90\)` | `\(10\)` | `\(10\)` | | `\(10\)` | `\(100\)` | `\(10\)` | `\(10\)` | ] .pull-right[  ] --- class: inverse, center, middle # Profits --- # Recall: The Firm's Two Problems .pull-left[ .smallest[ 1st Stage: .hi[firm's profit maximization problem]: 1. **Choose:** .hi-blue[ < output >] 2. **In order to maximize:** .hi-green[< profits >] 2nd Stage: .hi[firm's cost minimization problem]: 1. **Choose:** .hi-blue[ < inputs >] 2. **In order to _minimize_:** .hi-green[< cost >] 3. **Subject to:** .hi-red[< producing the optimal output >] - Minimizing costs `\(\iff\)` maximizing profits ] ] .pull-right[ .center[ ![](../images/management.jpg) ] ] --- # Visualizing Total Profit As `\(R(q)-C(q)\)` .pull-left[ - `\(\color{green}{\pi(q)}=\color{blue}{R(q)}-\color{red}{C(q)}\)` ] .pull-right[  ] --- # Visualizing Total Profit As `\(R(q)-C(q)\)` .pull-left[ - `\(\color{green}{\pi(q)}=\color{blue}{R(q)}-\color{red}{C(q)}\)` ] .pull-right[  ] --- # Visualizing Total Profit As `\(R(q)-C(q)\)` .pull-left[ - `\(\color{green}{\pi(q)}=\color{blue}{R(q)}-\color{red}{C(q)}\)` - Graph: find `\(q^*\)` to max `\(\pi \implies q^*\)` where max distance between `\(R(q)\)` and `\(C(q)\)` ] .pull-right[  ] --- # Visualizing Total Profit As `\(R(q)-C(q)\)` .pull-left[ - `\(\color{green}{\pi(q)}=\color{blue}{R(q)}-\color{red}{C(q)}\)` - Graph: find `\(q^*\)` to max `\(\pi \implies q^*\)` where max distance between `\(R(q)\)` and `\(C(q)\)` - Slopes must be equal: `$$\color{blue}{MR(q)}=\color{red}{MC(q)}$$` ] .pull-right[  ] --- # Visualizing Total Profit As `\(R(q)-C(q)\)` .pull-left[ - `\(\color{green}{\pi(q)}=\color{blue}{R(q)}-\color{red}{C(q)}\)` - Graph: find `\(q^*\)` to max `\(\pi \implies q^*\)` where max distance between `\(R(q)\)` and `\(C(q)\)` - Slopes must be equal: `$$\color{blue}{MR(q)}=\color{red}{MC(q)}$$` .smallest[ - At `\(q^*=5\)`: - `\(\color{blue}{R(q)=50}\)` - `\(\color{red}{C(q)=40}\)` - `\(\color{green}{\pi(q)=10}\)` ] ] .pull-right[  ] --- # Visualizing Profit Per Unit As `\(MR(q)\)` and `\(MC(q)\)` .pull-left[ - At low output `\(q<q^*\)`, can increase `\(\pi\)` by producing *more*: `\(\color{blue}{MR(q)}>\color{red}{MC(q)}\)` ] .pull-right[  ] --- # Visualizing Profit Per Unit As `\(MR(q)\)` and `\(MC(q)\)` .pull-left[ - At high output `\(q>q^*\)`, can increase `\(\pi\)` by producing *less*: `\(\color{blue}{MR(q)}<\color{red}{MC(q)}\)` ] .pull-right[  ] --- # Visualizing Profit Per Unit As `\(MR(q)\)` and `\(MC(q)\)` .pull-left[ - `\(\pi\)` is *maximized* where `\(\color{blue}{MR(q)}=\color{red}{MC(q)}\)` ] .pull-right[  ] --- class: inverse, center, middle # Comparative Statics --- # If Market Price Changes I .pull-left[ - Suppose the market price *increases* - Firm (always setting `\(MR=MC)\)` will respond by *producing more* ] .pull-right[  ] --- # If Market Price Changes II .pull-left[ - Suppose the market price *decreases* - Firm (always setting `\(MR=MC)\)` will respond by *producing more* ] .pull-right[  ] --- # If Market Price Changes II .pull-left[ - .hi-purple[The firm's marginal cost curve is its (inverse) supply curve].magenta[†] `$$Supply=MC(q)$$` - How it will supply the optimal amount of output in response to the market price - There is an exception to this! We will see shortly! ] .pull-right[  ] .footnote[.magenta[†] Mostly...there is an exception we will see shortly!] --- class: inverse, center, middle # Calculating Profit --- # Calculating Average Profit as `\(AR(q)-AC(q)\)` .pull-left[ - Profit is `$$\pi(q)=R(q)-C(q)$$` ] .pull-right[  ] --- # Calculating Average Profit as `\(AR(q)-AC(q)\)` .pull-left[ - Profit is `$$\pi(q)=R(q)-C(q)$$` - Profit per unit can be calculated as: `$$\begin{align*} \frac{\pi(q)}{q}&=\color{blue}{AR(q)}-\color{orange}{AC(q)}\\ &=\color{blue}{p}-\color{orange}{AC(q)}\\ \end{align*}$$` ] .pull-right[  ] --- # Calculating Average Profit as `\(AR(q)-AC(q)\)` .pull-left[ - Profit is `$$\pi(q)=R(q)-C(q)$$` - Profit per unit can be calculated as: `$$\begin{align*} \frac{\pi(q)}{q}&=\color{blue}{AR(q)}-\color{orange}{AC(q)}\\ &=\color{blue}{p}-\color{orange}{AC(q)}\\ \end{align*}$$` - Multiply by `\(q\)` to get total profit: `$$\pi(q)=q\left[\color{blue}{p}-\color{orange}{AC(q)} \right]$$` ] .pull-right[  ] --- # Calculating Average Profit as `\(AR(q)-AC(q)\)` .pull-left[ - At market price of .blue[p* = $10] - At q* = 5 (per unit): - At q* = 5 (totals): ] .pull-right[  ``` ## geom_segment: arrow = NULL, arrow.fill = NULL, lineend = butt, linejoin = round, na.rm = FALSE ## stat_identity: na.rm = FALSE ## position_identity ``` ] --- # Calculating Average Profit as `\(AR(q)-AC(q)\)` .pull-left[ - At market price of .blue[p* = $10] - At q* = 5 (per unit): - .blue[AR(5) = $10/unit] - At q* = 5 (totals): - .blue[R(5) = $50] ] .pull-right[  ] --- # Calculating Average Profit as `\(AR(q)-AC(q)\)` .pull-left[ - At market price of .blue[p* = $10] - At q* = 5 (per unit): - .blue[AR(5) = $10/unit] - .orange[AC(5) = $7/unit] - At q* = 5 (totals): - .blue[R(5) = $50] - .red[C(5) = $35] ] .pull-right[  ] --- # Calculating Average Profit as `\(AR(q)-AC(q)\)` .pull-left[ - At market price of .blue[p* = $10] - At q* = 5 (per unit): - .blue[AR(5) = $10/unit] - .orange[AC(5) = $7/unit] - .green[A`\\(\pi\\)`(5) = $3/unit] - At q* = 5 (totals): - .blue[R(5) = $50] - .red[C(5) = $35] - .green[`\\(\pi\\)` = $15] ] .pull-right[  ] --- # Calculating Average Profit as `\(AR(q)-AC(q)\)` .pull-left[ - At market price of .blue[p* = $2] - At q* = 1 (per unit): - At q* = 1 (totals): ] .pull-right[  ] --- # Calculating Average Profit as `\(AR(q)-AC(q)\)` .pull-left[ - At market price of .blue[p* = $2] - At q* = 1 (per unit): - .blue[AR(1) = $2/unit] - At q* = 1 (totals): - .blue[R(1) = $2] ] .pull-right[  ] --- # Calculating Average Profit as `\(AR(q)-AC(q)\)` .pull-left[ - At market price of .blue[p* = $2] - At q* = 1 (per unit): - .blue[AR(1) = $2/unit] - .orange[AC(1) = $10/unit] - At q* = 1 (totals): - .blue[R(1) = $2] - .red[C(1) = $10] ] .pull-right[  ] --- # Calculating Average Profit as `\(AR(q)-AC(q)\)` .pull-left[ - At market price of .blue[p* = $2] - At q* = 1 (per unit): - .blue[AR(1) = $2/unit] - .orange[AC(1) = $10/unit] - .green[A`\\(\pi\\)`(1) = -$8/unit] - At q* = 1 (totals): - .blue[R(1) = $2] - .red[C(1) = $10] - .green[`\\(\pi\\)`(1) = -$8] ] .pull-right[  ] --- class: inverse, center, middle # Short-Run Shut-Down Decisions --- # Short-Run Shut-Down Decisions .pull-left[ - What if a firm's profits at `\(q^*\)` are **negative** (i.e. it earns **losses**)? - Should it produce at all? ] .pull-right[ .center[ ![:scale 90%](../images/emptywarehouse.jpg) ] ] --- # Short-Run Shut-Down Decisions .pull-left[ - Suppose firm chooses to produce **nothing** `\((q=0)\)`: - If it has **fixed costs** `\((f>0)\)`, its profits are: `$$\begin{align*} \pi(q)&=pq-C(q)\\ \end{align*}$$` ] .pull-right[ .center[ ![:scale 90%](../images/emptywarehouse.jpg) ] ] --- # Short-Run Shut-Down Decisions .pull-left[ - Suppose firm chooses to produce **nothing** `\((q=0)\)`: - If it has **fixed costs** `\((f>0)\)`, its profits are: `$$\begin{align*} \pi(q)&=pq-C(q)\\ \pi(q)&=pq-f-VC(q)\\ \end{align*}$$` ] .pull-right[ .center[ ![:scale 90%](../images/emptywarehouse.jpg) ] ] --- # Short-Run Shut-Down Decisions .pull-left[ - Suppose firm chooses to produce **nothing** `\((q=0)\)`: - If it has **fixed costs** `\((f>0)\)`, its profits are: `$$\begin{align*} \pi(q)&=pq-C(q)\\ \pi(q)&=pq-f-VC(q)\\ \pi(0)&=-f\\ \end{align*}$$` ] .pull-right[ .center[ ![:scale 90%](../images/emptywarehouse.jpg) ] ] --- # Short-Run Shut-Down Decisions - A firm should choose to produce nothing `\((q=0)\)` only when: `$$\begin{align*} \pi \text{ from producing} &< \pi \text{ from not producing}\\ \end{align*}$$` --- # Short-Run Shut-Down Decisions - A firm should choose to produce nothing `\((q=0)\)` only when: `$$\begin{align*} \pi \text{ from producing} &< \pi \text{ from not producing}\\ \pi(q) &< -f \\ \end{align*}$$` --- # Short-Run Shut-Down Decisions - A firm should choose to produce nothing `\((q=0)\)` only when: `$$\begin{align*} \pi \text{ from producing} &< \pi \text{ from not producing}\\ \pi(q) &< -f \\ pq-VC(q)-f &<-f\\ \end{align*}$$` --- # Short-Run Shut-Down Decisions - A firm should choose to produce nothing `\((q=0)\)` only when: `$$\begin{align*} \pi \text{ from producing} &< \pi \text{ from not producing}\\ \pi(q) &< -f \\ pq-VC(q)-f &<-f\\ pq-VC(q) &< 0\\ \end{align*}$$` --- # Short-Run Shut-Down Decisions - A firm should choose to produce nothing `\((q=0)\)` only when: `$$\begin{align*} \pi \text{ from producing} &< \pi \text{ from not producing}\\ \pi(q) &< -f \\ pq-VC(q)-f &<-f\\ pq-VC(q) &< 0\\ pq &< VC(q)\\ \end{align*}$$` --- # Short-Run Shut-Down Decisions - A firm should choose to produce nothing `\((q=0)\)` only when: `$$\begin{align*} \pi \text{ from producing} &< \pi \text{ from not producing}\\ \pi(q) &< -f \\ pq-VC(q)-f &<-f\\ pq-VC(q) &< 0\\ pq &< VC(q)\\ \color{red}{p} & \color{red}{< AVC(q)}\\ \end{align*}$$` --- # Short-Run Shut-Down Decisions .pull-left[ - .hi[Shut down price]: firm will shut down production *in the short run* when `\(p<AVC(q)\)` ] .pull-right[ .center[ ![:scale 100%](../images/shutitdown.jpg) ] ] --- class: inverse, center, middle # The Firm's Short Run Supply Decision --- # The Firm's Short Run Supply Decision .pull-left[  ] --- # The Firm's Short Run Supply Decision .pull-left[  ] --- # The Firm's Short Run Supply Decision .pull-left[  ] --- # The Firm's Short Run Supply Decision .pull-left[  ] .pull-right[ ] --- # The Firm's Short Run Supply Decision .pull-left[  ] .pull-right[ ] --- # The Firm's Short Run Supply Decision .pull-left[  ] .pull-right[ .center[ Firm's .red[short run (inverse) supply]: ] `$$\begin{cases} p=MC(q) & \text{if } p \geq AVC \\ q=0 & \text{If } p < AVC\\ \end{cases}$$` ] --- # The Firm's Short Run Supply Decision .pull-left[  ] .pull-right[ .center[ Firm's .red[short run (inverse) supply]: ] `$$\begin{cases} p=MC(q) & \text{if } p \geq AVC \\ q=0 & \text{If } p < AVC\\ \end{cases}$$` ] --- # Summary: **1. Choose `\(q^*\)` such that `\(MR(q)=MC(q)\)`** -- **2. Profit `\(\pi=q[p-AC(q)]\)`** -- **3.  The current market price of a haircut is $15. Bob's daily short run costs are given by: `$$\begin{align*} C(q) &= 0.5q^2\\ MC(q) &=q\\ \end{align*}$$` ] 1. How many haircuts per day would maximize Bob's profits? 2. How much profit will Bob earn per day? 3. Find Bob's shut down price. 4. Write Bob's short-run inverse supply function.


Looking at the navigation above the form, you see a number of links. The first is New Posts. What letter comes after F and A in FA_?


| PERL5302DELTA(1) | Perl Programmers Reference Guide | PERL5302DELTA(1) |
If you are upgrading from an earlier release such as 5.30.0, first read perl5301delta, which describes differences between 5.30.0 and 5.30.1.
[GH #17221 <https://github.com/Perl/perl5/issues/17221>]
[GH #17218 <https://github.com/Perl/perl5/issues/17218>]
[GH #17370 <https://github.com/Perl/perl5/issues/17370>]
[GH #17372 <https://github.com/Perl/perl5/issues/17372>]
[GH #17490 <https://github.com/Perl/perl5/issues/17490>]
Excluding auto-generated files, documentation and release tools, there were approximately 920 lines of changes to 30 .pm, .t, .c and .h files.
Perl continues to flourish into its fourth decade thanks to a vibrant community of users and developers. The following people are known to have contributed the improvements that became Perl 5.30.2:
Chris 'BinGOs' Williams, Dan Book, David Mitchell, Hugo van der Sanden, Karen Etheridge, Karl Williamson, Matthew Horsfall, Nicolas R., Petr PisaX, Renee Baecker, Sawyer X, Steve Hay, Tomasz Konojacki, Tony Cook, Yves Orton.
The list above is almost certainly incomplete as it is automatically generated from version control history. In particular, it does not include the names of the (very much appreciated) contributors who reported issues to the Perl bug tracker.
Many of the changes included in this version originated in the CPAN modules included in Perl's core. We're grateful to the entire CPAN community for helping Perl to flourish.
For a more complete list of all of Perl's historical contributors, please see the AUTHORS file in the Perl source distribution.
If you believe you have an unreported bug, please open an issue at <https://github.com/Perl/perl5/issues>. Be sure to trim your bug down to a tiny but sufficient test case.
If the bug you are reporting has security implications which make it inappropriate to send to a public issue tracker, then see "SECURITY VULNERABILITY CONTACT INFORMATION" in perlsec for details of how to report the issue.
perlthanks
This will send an email to the Perl 5 Porters list with your show of thanks.
The INSTALL file for how to build Perl.
The README file for general stuff.
The Artistic and Copying files for copyright information.
| 2020-05-14 | perl v5.30.3 |


The "end game" of a Technical Specification (TS) is to have the features in there added to a future version of the C++ standard. When this happens, the TS can be retired. Sometimes, only part of at TS is added to the standard, and the rest of the features may be incorporated into the next version of the TS.
Adoption leaves library implementors with two implementations of a feature, one in namespace std, and the other in namespace std::experimental. The first one will continue to evolve (via issues and papers), while the other will not. Gradually they will diverge. It's not good for users to have two (subtly) different implementations of the same functionality in the same library.
As features are adopted into the main standard, we will implement them in namespace std, and then remove the versions in std::experimental. The removal will not happen immediately, because that would be unhelpful for users - giving them no chance to update their code.
The rule of thumb that libc++ will follow is: one year.
One year after we ship an implementation of a feature in std, we will remove it from std::experimental.
A specific example: The first release of clang/libc++ that officially supported C++17 was 5.0. For the 7.0 release (one year after 5.0), we will remove the features that were adopted into C++17 from the TSes, and that were present in namespace std in the 5.0 release.
Library Fundamentals TS V1 and V2
Most (but not all) of the features of the LFTS were accepted into C++17.
| Section | Feature | shipped in std | To be removed from std::experimental | Notes |
|---|---|---|---|---|
| 2.1 | uses_allocator construction | |||
| 3.1.2 | erased_type | Not part of C++17 | ||
| 3.2.1 | tuple_size_v | Removed | ||
| 3.2.2 | apply | Removed | ||
| 3.3.1 | All of the '_v' traits in <type_traits> | Removed | ||
| 3.3.2 | invocation_type and raw_invocation_type | Not part of C++17 | ||
| 3.3.3 | Logical operator traits | Removed | ||
| 3.3.3 | Detection Idiom | Only partially in C++17 | ||
| 3.4.1 | All of the '_v' traits in <ratio> | Removed | ||
| 3.5.1 | All of the '_v' traits in <chrono> | Removed | ||
| 3.6.1 | All of the '_v' traits in <system_error> | Removed | ||
| 3.7 | propagate_const | Not part of C++17 | ||
| 4.2 | Enhancements to function | |||
| 4.3 | searchers | |||
| 5 | optional | Removed | ||
| 6 | any | Removed | ||
| 7 | string_view | Removed | ||
| 8.2.1 | shared_ptr enhancements | |||
| 8.2.2 | weak_ptr enhancements | |||
| 8.5 | memory_resource | |||
| 8.6 | polymorphic_allocator | |||
| 8.7 | resource_adaptor | Not part of C++17 | ||
| 8.8 | Access to program-wide memory_resource objects | |||
| 8.9 | Pool resource classes | |||
| 8.10 | monotonic_buffer_resource | |||
| 8.11 | Alias templates using polymorphic memory resources | |||
| 8.12 | Non-owning pointers | Not part of C++17 | ||
| 11.2 | promise | Not part of C++17 | ||
| 11.3 | packaged_task | Not part of C++17 | ||
| 12.2 | search | |||
| 12.3 | sample | Removed | ||
| 12.4 | shuffle | Not part of C++17 | ||
| 13.1 | gcd and lcm | Removed | ||
| 13.2 | Random number generation | Not part of C++17 | ||
| 14 | Reflection Library | Not part of C++17 |
FileSystem TS
The FileSystem TS was accepted (in toto) for C++17.
The FileSystem TS implementation is still (as of v6.0) in namespace std::experimental.
Parallelism TS V1 and V2
Some (most) of the Parallelism TS was accepted for C++17.
We have not yet shipped an implementation of the Parallelism TS.
Coroutines TS
The Coroutines TS is not yet part of a shipping standard.
We are shipping (as of v5.0) an implementation of the Coroutines TS in namespace std::experimental.
Networking TS
The Networking TS is not yet part of a shipping standard.
We have not yet shipped an implementation of the Networking TS.
Ranges TS
The Ranges TS is not yet part of a shipping standard.
We have not yet shipped an implementation of the Ranges TS.
Concepts TS
The Concepts TS is not yet part of a shipping standard, but it has been adopted into the C++20 working draft.
We have not yet shipped an implementation of the Concepts TS.
Concurrency TS
None of the Concurrency TS was accepted for C++17.
We have not yet shipped an implementation of the Concurrency TS.
Last Updated: 8-Feb-2018


Society of Physics Students
Undergraduate physics club @ UCLA |
Facebook
|
Twitter
Toggle navigation
SPS
Home
Updates
About
Gallery
Officers
2016-2017
2015-2016
2014-2015
Documents
•Arduino
•Logistics
SPS
Updates & Announcements
No older posts :p
← Older
Newer →


| blockchain |
getbestblockhash |
| blockchain |
getblock |
| blockchain |
getblockchaininfo |
| blockchain |
getblockcount |
| blockchain |
getblockhash |
| blockchain |
getblockheader |
| blockchain |
getchaintips |
| blockchain |
getchaintxstats |
| blockchain |
getdifficulty |
| blockchain |
getmempoolancestors |
| blockchain |
getmempooldescendants |
| blockchain |
getmempoolentry |
| blockchain |
getmempoolinfo |
| blockchain |
getrawmempool |
| blockchain |
gettxout |
| blockchain |
gettxoutproof |
| blockchain |
gettxoutsetinfo |
| blockchain |
preciousblock |
| blockchain |
pruneblockchain |
| blockchain |
savemempool |
| blockchain |
verifychain |
| blockchain |
verifytxoutproof |
| control |
getmemoryinfo |
| control |
help |
| control |
logging |
| control |
stop |
| control |
uptime |
| generating |
generate |
| generating |
generatetoaddress |
| hidden |
addwitnessaddress |
| hidden |
echo |
| hidden |
echojson |
| hidden |
estimaterawfee |
| hidden |
getinfo |
| hidden |
invalidateblock |
| hidden |
reconsiderblock |
| hidden |
resendwallettransactions |
| hidden |
setmocktime |
| hidden |
syncwithvalidationinterfacequeue |
| hidden |
waitforblock |
| hidden |
waitforblockheight |
| hidden |
waitfornewblock |
| mining |
getblocktemplate |
| mining |
getmininginfo |
| mining |
getnetworkhashps |
| mining |
prioritisetransaction |
| mining |
submitblock |
| network |
addnode |
| network |
clearbanned |
| network |
disconnectnode |
| network |
getaddednodeinfo |
| network |
getconnectioncount |
| network |
getnettotals |
| network |
getnetworkinfo |
| network |
getpeerinfo |
| network |
listbanned |
| network |
ping |
| network |
setban |
| network |
setnetworkactive |
| rawtransactions |
combinerawtransaction |
| rawtransactions |
createrawtransaction |
| rawtransactions |
decoderawtransaction |
| rawtransactions |
decodescript |
| rawtransactions |
fundrawtransaction |
| rawtransactions |
getrawtransaction |
| rawtransactions |
sendrawtransaction |
| rawtransactions |
signrawtransaction |
| test |
rpcNestedTest |
| util |
createmultisig |
| util |
estimatefee |
| util |
estimatesmartfee |
| util |
signmessagewithprivkey |
| util |
validateaddress |
| util |
verifymessage |
| wallet |
abandontransaction |
| wallet |
abortrescan |
| wallet |
addmultisigaddress |
| wallet |
backupwallet |
| wallet |
bumpfee |
| wallet |
dumpprivkey |
| wallet |
dumpwallet |
| wallet |
encryptwallet |
| wallet |
getaccount |
| wallet |
getaccountaddress |
| wallet |
getaddressesbyaccount |
| wallet |
getbalance |
| wallet |
getnewaddress |
| wallet |
getrawchangeaddress |
| wallet |
getreceivedbyaccount |
| wallet |
getreceivedbyaddress |
| wallet |
gettransaction |
| wallet |
getunconfirmedbalance |
| wallet |
getwalletinfo |
| wallet |
importaddress |
| wallet |
importmulti |
| wallet |
importprivkey |
| wallet |
importprunedfunds |
| wallet |
importpubkey |
| wallet |
importwallet |
| wallet |
keypoolrefill |
| wallet |
listaccounts |
| wallet |
listaddressgroupings |
| wallet |
listlockunspent |
| wallet |
listreceivedbyaccount |
| wallet |
listreceivedbyaddress |
| wallet |
listsinceblock |
| wallet |
listtransactions |
| wallet |
listunspent |
| wallet |
listwallets |
| wallet |
lockunspent |
| wallet |
move |
| wallet |
removeprunedfunds |
| wallet |
rescanblockchain |
| wallet |
sendfrom |
| wallet |
sendmany |
| wallet |
sendtoaddress |
| wallet |
setaccount |
| wallet |
settxfee |
| wallet |
signmessage |
| wallet |
walletlock |
| wallet |
walletpassphrase |
| wallet |
walletpassphrasechange |


| Setting | Value | |||
|---|---|---|---|---|
| Documentation | RobotFramework Eclipse Library provides keywords to drive Eclipse test cases.\n It uses SWTBot Open Source library to interact with Eclipse widgets. |
|||
| Library | EclipseLibrary | |||
| Keyword | Action | Arguments | ||
|---|---|---|---|---|
| Click On Button | [Arguments] | ${mnemonic} | @{index} | |
| Find Button | ${mnemonic} | @{index} | ||
| Click Button | ||||
| Button Should Be Active | [Arguments] | ${mnemonic} | @{index} | |
| [Documentation] | both mnemonic or index are optional\n If only one parameter is given and if it is numeric, it will be used as index\n |
|||
| Find Button | ${mnemonic} | @{index} | ||
| ${_bool_} = | Is Button Active | |||
| Should Be True | ${_bool_} | |||
| Button Should Not Be Active | [Arguments] | ${mnemonic} | @{index} | |
| Find Button | ${mnemonic} | @{index} | ||
| ${_bool_} = | Is Button Active | |||
| Should Not Be True | ${_bool_} | |||
| Button Should Be Visible | [Arguments] | ${mnemonic} | @{index} | |
| [Documentation] | both mnemonic or index are optional\n If only one parameter is given and if it is numeric, it will be used as index\n |
|||
| Find Button | ${mnemonic} | @{index} | ||
| ${_bool_} = | Is Button Visible | |||
| Should Be True | ${_bool_} | |||
| Button Should Not Be Visible | [Arguments] | ${mnemonic} | @{index} | |
| [Documentation] | both mnemonic or index are optional\n If only one parameter is given and if it is numeric, it will be used as index\n |
|||
| Find Button | ${mnemonic} | @{index} | ||
| ${_bool_} = | Is Button Visible | |||
| Should Not Be True | ${_bool_} | |||
| The view ${view_name} is opened | Find View By Title | ${view_name} | ||
| Show View | ||||
| ${bool} | Is View Active | |||
| Should Be True | ${bool} | |||
| Start Eclipse with parameters | [Arguments] | ${install}=file:/d:/eclipse/eclipse35 | ${data}=file:/d:/bootstrap35-sv3 | |
| Start Eclipse | -consolelog | -debug | -install | |
| ... | ${install} | -data | ${data} | |
| ... | -nosplash | |||
| Set Capture Screenshot On Error | screenshot_error.jpg | |||
| The perspective ${perspective_name} is opened | Find Perspective By Label | ${perspective_name} | ||
| Activate Perspective | ||||
| ${bool} | Is Perspective Active | |||
| Should Be True | ${bool} | |||
| Label Should Be Enabled | [Arguments] | ${mnemonic} | @{index} | |
| Find Label | ${mnemonic} | @{index} | ||
| ${bool}= | Is Label Enabled | |||
| Should Be True | ${bool} | |||
| Label Should Be Disabled | [Arguments] | ${mnemonic} | @{index} | |
| Find Label | ${mnemonic} | @{index} | ||
| ${bool}= | Is Label Enabled | |||
| Should Not Be True | ${bool} | |||
| Button Should Be Enabled | [Arguments] | ${mnemonic} | @{index} | |
| [Documentation] | both mnemonic or index are optional\n If only one parameter is given and if it is numeric, it will be used as index\n |
|||
| Find Button | ${mnemonic} | @{index} | ||
| ${_bool_} = | Is Button Enabled | |||
| Should Be True | ${_bool_} | |||
| Button Should Be Disabled | [Arguments] | ${mnemonic} | @{index} | |
| [Documentation] | both mnemonic or index are optional\n If only one parameter is given and if it is numeric, it will be used as index\n |
|||
| Find Button | ${mnemonic} | @{index} | ||
| ${_bool_} = | Is Button Enabled | |||
| Should Not Be True | ${_bool_} | |||
| Stop Eclipse | Find Menu | File | ||
| Click Menu | ||||
| Find Sub Menu | Exit | |||
| Click Menu | ||||


The subtraction on floating point values which is immediately cast to integral type is also warned as precision loss is possible due to rounding.
The warning is not issued when it's statically determined that value ranges are limited, and overflow never occurs. Additionally subtraction on int numbers greater than or equal to 0 will never overflow. Therefore this inspection tries not to warn in those cases. Methods that always return zero or greater can be specified in this inspection's settings.
Use the table below to specify methods which are safe to use inside a subtraction. Methods are safe when they return an int value that is always greater than or equal to 0. Alternatively you may use javax.annotation.Nonnegative annotation on such methods.


/**
* This smart contract code is Copyright 2017 TokenMarket Ltd. For more information see https://tokenmarket.net
*
* Licensed under the Apache License, version 2.0: https://github.com/TokenMarketNet/ico/blob/master/LICENSE.txt
*/
pragma solidity ^0.4.8;
import "./StandardTokenExt.sol";
/**
* Define interface for releasing the token transfer after a successful crowdsale.
*/
contract ReleasableToken is StandardTokenExt {
/* The finalizer contract that allows unlift the transfer limits on this token */
address public releaseAgent;
/** A crowdsale contract can release us to the wild if ICO success. If false we are are in transfer lock up period.*/
bool public released = false;
/** Map of agents that are allowed to transfer tokens regardless of the lock down period. These are crowdsale contracts and possible the team multisig itself. */
mapping (address => bool) public transferAgents;
/**
* Limit token transfer until the crowdsale is over.
*
*/
modifier canTransfer(address _sender) {
Iif(!released) {
if(!transferAgents[_sender]) {
throw;
}
}
_;
}
/**
* Set the contract that can call release and make the token transferable.
*
* Design choice. Allow reset the release agent to fix fat finger mistakes.
*/
function setReleaseAgent(address addr) onlyOwner inReleaseState(false) public {
// We don't do interface check here as we might want to a normal wallet address to act as a release agent
releaseAgent = addr;
}
/**
* Owner can allow a particular address (a crowdsale contract) to transfer tokens despite the lock up period.
*/
function setTransferAgent(address addr, bool state) onlyOwner inReleaseState(false) public {
transferAgents[addr] = state;
}
/**
* One way function to release the tokens to the wild.
*
* Can be called only from the release agent that is the final ICO contract. It is only called if the crowdsale has been success (first milestone reached).
*/
function releaseTokenTransfer() public onlyReleaseAgent {
released = true;
}
/** The function can be called only before or after the tokens have been releasesd */
modifier inReleaseState(bool releaseState) {
Iif(releaseState != released) {
throw;
}
_;
}
/** The function can be called only by a whitelisted release agent. */
modifier onlyReleaseAgent() {
Iif(msg.sender != releaseAgent) {
throw;
}
_;
}
function transfer(address _to, uint _value) canTransfer(msg.sender) returns (bool success) {
// Call StandardToken.transfer()
return super.transfer(_to, _value);
}
function transferFrom(address _from, address _to, uint _value) canTransfer(_from) returns (bool success) {
// Call StandardToken.transferForm()
return super.transferFrom(_from, _to, _value);
}
} |


You have clicked on {{categoryName}} {{name}}.
Select which category you want to inspect to:
You cannot inspect further!
You have clicked on {{categoryName}} {{name}}.
Select which category you want to group the data of {{name}} by:
Select which {{compareForm.controls.category.value.plural}} do you want to group your data by:
Loading...
You have clicked on {{categoryName}} {{name}}.
Click 'Go to details page' button to monitor success criteria in detail.
You have clicked on {{categoryName}} {{name}}.
Select which {{categoryNamePlural}} do you want to compare:
You have clicked on {{categoryName}} {{name}}.
Click 'Go to timeline' button to monitor the temporal evolution.


STAR TREK: GENERATIONS A film review by Scott Renshaw Copyright 1994 Scott Renshaw
Starring: Patrick Stewart, Malcolm McDowell, William Shatner, Jonathan Frakes, Brent Spiner. Screenplay: Ronald D. Moore and Brannon Braga. Director: David Carson.
Whether or not I would be considered a Trekker probably depends on whom you would ask. I have been a fan of both the original series and its recently retired follow-up, as well as the even- numbered entries in the film series. However, I have never been one of those folks who store away Trek minutiae and get into debates over the relative merits of Spock vs. Data. Somewhere along the line, the "Star Trek" film series began to seem more and more directed at those in the latter category, and STAR TREK: GENERATIONS may be the natural conclusion of this direction. Its production values may be high, but the writing is frequently appalling, and instead of a script has a collection of references, in-jokes and ill-defined characters.
GENERATIONS opens in the late 23rd Century, where members of the original Enterprise crew, including Captain James T. Kirk (William Shatner), are present for the christening of the latest ship to bear that name. No sooner is it out on its maiden voyage than a distress signal brings them to the Nexus, a mysterious ribbon of energy. Among those rescued from the Nexus is long-lived alien Dr. Soran (Malcolm McDowell), but he is none-too-pleased at being back in the real world. Seventy-eight years later, Soran is still trying to get back to the Nexus, and again encounters an Enterprise, this one led by Captain Jean-Luc Picard (Patrick Stewart). Soran's plan involves destroying a star with an inhabited planet, and the only hope for saving 230 million people is the historic meeting of two Enterprise captains.
I don't think it is an insult to fans of "Star Trek" to suggest that to a certain extent, the particulars of a plot are not really the most important elements in a "Trek" film. It's about a chance to visit with old friends, and if they are involved in a truly interesting story it can be considered gravy. But even that interpretation assumes that characterization has to be consistent, and in GENERATIONS that just doesn't happen. Data (Brent Spiner) is particularly victimized by this sloppy writing. In the film's major sub-plot, Data decides to use a chip which will give him human emotions, a chip he has had in his possession for over a year. What is his motivation for taking this drastic and perhaps dangerous step? He doesn't get a joke. Once the chip is in place, Spiner gets to have a lot of fun with Data's new emotions, but the point is that he's no longer the Data we know. If the rest of the "Next Generation" cast fares any better, it's only because they're on the screen so little that they're simply window dressing.
But then again, this story isn't really about the new crew. It's about Kirk and Picard, the cowboy and the politician, sharing the screen. The huge surprise is that William Shatner blows Patrick Stewart away. Perhaps because he knows this is his last go-round in the part that made him a legend, Shatner looks like he's having the time of his life; Stewart, on the other hand, borders on the deferrential, and is saddled with the same lackluster motivations as everyone else in the cast. Stewart's edginess is indicative of what may be a very bad sign for Paramount's hopes to turn the new cast into the same kind of franchise the original cast was: these are clearly the Not Ready for Big Screen Players. By contrast, the prologue which features Scotty (James Doohan) and Chekhov (Walter Koenig) is the highlight of the film. These characters have become part of the popular mythology, and when they take over the Enterprise in a moment of distress, it's a moment of high energy. Only thirty years of history can create a moment like that, and that's time this new crew doesn't have.
It certainly doesn't help that the plot they are stuck in is so convoluted and badly constructed that you need to leave a trail of bread crumbs to find your way out again. It's also loaded with little gags all aimed at those who are familiar enough with the series to turn to all their friends in a shared nudge of recognition. I wonder whether anyone who is coming in cold to the "Trek" universe at this point would be anything but baffled and bored by what is going on. And what a waste of Malcolm McDowell, who could have been a great monomanical villain but should have been given a much more compelling reason for his obsession.
There are plenty of details both scientific and of continuity to pick at, but to do so would be to ignore GENERATIONS' major flaws. However, I would like to make one recommendation: seatbelts. Not just for the Enterprise crew, although you'd think by the 24th Century space travel wouldn't involve rolling around on the floor during a battle. No, I think the audience might need them more. This journey is one bumpy ride.
On the Renshaw scale of 0 to 10 GENERATIONS wrecks: 4.
.
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


Search: Search by : Select an option Faculty Department Class Title of Activity Student Specify name : Result: Title Faculty Students Departmemt Class Role Email-ID Contact Title1 Faculty1 Students1 Departmemt1 Class1 Role1 Email-ID1 Contact1 Students2 Departmemt2 Class2 Role2 Email-ID2 Contact2 Students3 Departmemt3 Class3 Role3 Email-ID3 Contact3 Title2 Faculty2 Students1 Departmemt1 Class1 Role1 Email-ID1 Contact1 Students2 Departmemt2 Class2 Role2 Email-ID2 Contact2 Students3 Departmemt3 Class3 Role3 Email-ID3 Contact3 Students4 Departmemt4 Class4 Role4 Email-ID4 Contact4
Search by : Select an option Faculty Department Class Title of Activity Student Specify name :
Title Faculty Students Departmemt Class Role Email-ID Contact Title1 Faculty1 Students1 Departmemt1 Class1 Role1 Email-ID1 Contact1 Students2 Departmemt2 Class2 Role2 Email-ID2 Contact2 Students3 Departmemt3 Class3 Role3 Email-ID3 Contact3 Title2 Faculty2 Students1 Departmemt1 Class1 Role1 Email-ID1 Contact1 Students2 Departmemt2 Class2 Role2 Email-ID2 Contact2 Students3 Departmemt3 Class3 Role3 Email-ID3 Contact3 Students4 Departmemt4 Class4 Role4 Email-ID4 Contact4


Overview
Welcome to the Backup & Restore documentation for Bluemix Dedicated.
Not every service on Bluemix has a backup strategy because not every service manages data to be backed up. What’s more, given the approach on Bluemix of letting each product operate in a way consistent with the standard operational principles of that underlying service, the backup and restore procedures for each can differ. The following services have specific backup and restore or high availability procedures.
DashDB – For the DashDB Managed Service only, encrypted backups on the full database are done daily. The last two backups are retained. Point-in-time restores are not available.
Apache Spark – You can reuse an external SoftLayer Object Storage account in Apache Spark on Bluemix to store data redundantly. Softlayer Object storage is highly available and replicated.
DB2 on Cloud - DB2 on Cloud allows you to perform regular backups of your data to remote cloud storage (IBM Softlayer Object Storage or Amazon S3) using the BACKUP command.
Some Bluemix services (such as the Time Series Database) do not have customer-accessible backup or restore procedures. The backup and restore procedures of third-party services such as ElephantSQL are the responsibility of the service provider.
Compose
Relationship to Bluemix
The Compose DBaaS is an IBM company. It is not directly associated with Cloud Foundry (the open-source project behind IBM Bluemix). Compose provides Databases-as-a-Service (aka: DBaaS) for popular open source databases. The service uses a Bluemix Service Broker to bind the Compose service to the Bluemix Platform.
Backup
By default, the Compose DBaaS does scheduled backups for every database deployment.
Application Owners
As an application owner, scheduled backup does provide a safety net that will generally suffice for applications that make use of a single database. There are use cases, such as an application that uses multiple databases, that may necessitate backing up manually.
Bluemix Administrators
As an administrator of the Bluemix environment, Compose’s **Scheduled Backups ensure that databases are being backed up without requiring additional administrative attention. There are occasions where a Bluemix Administrator may want to create a backup [On Demand] or have the need to capture a consistent state of the entire system. These topics are covered below.
Design
Backups done by Compose were designed to minimize, or eliminate, any interruption to the database performance. To achieve this, backups are run against a non-active member of a deployment such that they will not be running against a node that serves queries.
For example, when a new Compose MongoDB deployment is created, part of the deployment is the addition of a full (but hidden) third member to the MongoDB replica set. The hidden member node is paused from accepting writes by sending it a db.fsyncLock()
which flushes out all the pending write operations and locks the database.
pg_dump -U admin -h bluemix-test1-ibm-305.2.compose.direct -p 14047 -d exampledb > db.sql
All Postgres Databases
To create a back up of all databases in a particular Postgres deployment, use the pg_dumpall
tool on your localhost, execute the following command:
pg_dump -U admin -h bluemix-test1-ibm-305.2.compose.direct -p 14047 > db.out
MongoDB
- Login to Bluemix
- On the Bluemix left-hand navigation, click the Services link and a specific Compose Service deployment.
3. From the Compose service dashboard in Bluemix click the Launch button to get to the Compose dashboard.
4. From the Compose service, select the target MongoDB deployment to access the Overview dashboard.
5. Select a Mongo database and then select the Users
view.
6. Verify that you have an existing user with readOnly
, readWrite
, or dbOwner
. If not, add a user.
7. With valid user credentials on your target database, follow the instructions using mongodump
as described in MongoDB’s Back Up a Sharded Cluster with Database Dumps documentation.
Elasticsearch
Elasticsearch does not provide a backup utility; however node.js community members created elasticdump
. In order to use elasticdump
, you must have Node.JS and NPM installed. Binaries for installing Node and NPM are available at https://nodejs.org/download/release/latest/.
npm install -g elasticdump
2. Login to Bluemix
3. On the Bluemix left-hand navigation, click the Services link and a specific Compose Service deployment.
4. From the Compose service dashboard in Bluemix click the Launch button to get to the Compose dashboard.
5. From the Compose service, select the target Elasticsearch deployment to access the Overview dashboard. Note the connection string
6. From the Overview page, select the Users
view and confirm that you have a valid and current user credentials.
7. In order for the elasticdump
tool to create a backup of all indexes, the base URL must be specified as an --input
and a directory for --output
, as directed in the MultiElasticDump section of the documentation.
elasticdump --input=https://admin:Passw0rd@sl-eu-lon-2-portal1.dblayer.com:10331/ --output=/tmp/
Redis
- Login to Bluemix
- On the Bluemix left-hand navigation, click the Services link and a specific Compose Service deployment.
3. From the Compose service dashboard in Bluemix click the Launch button to get to the Compose dashboard.
4. From the Compose service, select the target Redis deployment to access the Overview dashboard.
5. Scroll down on the Overview dashboard until you reach the Connection Info
section. Use these credentials with the redis-cli
tool from your local host.
6. From your local host execute the following command to create a remote backup of Redis
redis-cli -h <CONNECTION> -p 14250 -a <PASSWORD> --rdb dump.rdb
RethinkDB
- Login to Bluemix
- On the Bluemix left-hand navigation, click the Services link and a specific Compose Service deployment.
3. From the Compose service dashboard in Bluemix click the Launch button to get to the Compose dashboard.
4. From the Compose service, select the target Rethink deployment to access the Overview dashboard.
5. From the Rethink Overview, scroll down to the Connection Info
section and click Show
and note the password. Then scroll down to the SSL Certificate (Self-Signed)
section.
6. Copy the contents of the SSL certificate to a file (ie: compose.rethinkdb.crt
) on your local host
7. From your local host, execute the rethinkdb dump
command with the host, port, --tls-cert
and the -p
(password) flags:
rethinkdb dump -p --tls-cert compose.rethinkdb.crt -c sl-us-dal-9-portal.1.dblayer.com:10828
Enter your password when prompted and a successful result will output something similar to the following:
[========================================] 100%
9342 rows exported from 1 tables, with 0 secondary indexes
Done (0.75 seconds)
Zipping export directory...
Done (0 seconds)
From Bluemix
In theory, it is possible to be able to execute binary backup tools, such as mongodump
or pg_dump
from Bluemix. In practice, it can be a complex task, as it requires creating a custom buildpack to build the binary from source. Additionally, the binary must be built with SSL/TLS support, since connecting to Compose requires an SSL certificate. Due to the SSL requirement, the binary should be built with the openssl libraries from the Bluemix host, and not from any other host.
A backup from Bluemix may reduce the network latency from Compose in comparison to running backup commands from your local network; however, storing data on the filesystem of a Bluemix application is strongly discouraged and should be immediately stored in secure location, such as an object store. Moreover, additional consideration will have to be given to coordination, if there are more than one application instances running of the backup process.
Multiple Database Backup
The primary reason to do manual backups of a Compose database, is to coordinate a backup across multiple database in order to preserve a consistent state. The application architecture determines the complexity of making consistent backups across databases. A common technique to ensure a consistent backup across a distributed system is to make data immutable and versioned.
Immutability and Versioning
// `crypto` provides the consistent hash
const crypto = require('crypto');
const hash = crypto.createHash('sha256');
// `mongoose` provides an Object Model for MongoDB
var mongoose = require('mongoose')
, Schema = mongoose.Schema
// Providing a model for version history of any object
, versionModel = new Schema({
version: String,
data: Schema.Types.Mixed
})
, Version = mongoose.model('Version', versionModel)
// An example object model
, userModel = new Schema({
first_name: String,
middle_name: String,
last_name: String,
role: String,
dob: Date,
version: String,
history: [Version]
})
, User = mongoose.model('User', userModel);
// Connect to Mongo
mongoose.connect(process.env.MONGO_URI, {
user: process.env.MONGO_USER,
pass: process.env.MONGO_PASS
});
// an example document to populate the object model
var user = {
first_name: 'Sidney',
middle_name: 'R.',
last_name: 'Garfield',
role: 'physician',
dob: new Date(1906, 04, 17)
}
// crypto requires a string or buffer (ie: `json`)
, json = JSON.stringify(record)
, doc = new User(user)
, version = new Version({
data: user,
// `hex` will marshall a string to fit `Version` schema
version: hash.update(json).digest('hex')
});
// Saves the document with the version history and the latest version
doc.version = version.version;
doc.history.push(version);
doc.save();
package com.ibm.immutability.example;
import com.mongodb.MongoClient;
import org.mongodb.morphia.Datastore;
// `Morphia` provides an Object Model for MongoDB
import org.mongodb.morphia.Morphia;
import org.mongodb.morphia.annotations.Entity;
import org.mongodb.morphia.annotations.Field;
import org.mongodb.morphia.annotations.Id;
import org.mongodb.morphia.annotations.Index;
import org.mongodb.morphia.annotations.Indexes;
import org.mongodb.morphia.annotations.Property;
import java.util.ArrayList;
import java.util.List;
// `java.security.MessageDigest` provides the consistent hash
import java.security.MessageDigest;
// JSON utils
import org.codehaus.jackson.JsonGenerationException;
import org.codehaus.jackson.map.JsonMappingException;
import org.codehaus.jackson.map.ObjectMapper;
public final class ImmutabilityDemo {
private ImmutabilityDemo(){
}
public static void main(final String[] args) {
final Morphia morphia = new Morphia();
// tell morphia where to find your classes
// can be called multiple times with different packages or classes
morphia.mapPackage("org.ibm.immutability.example");
// create the Datastore connecting to the database running on the default port on the local host
final Datastore datastore = morphia.createDatastore(new MongoClient(), "immutability_example");
SimpleDateFormat ft = new SimpleDateFormat ("yyyy-MM-dd");
Date dob;
dob = ft.parse("1906-04-17");
final User user = new User("Sidney", "R.", "Garfiled", "physician", dob);
// an example document to populate the object model
datastore.save(user);
}
}
// Provides an example User model object
@Entity("users")
class User {
@Id
private ObjectId id;
private String firstName;
private String middleName;
private String lastName;
private String role;
private Date dob;
private String version;
@Reference
private List<Version> history = new ArrayList<Version>();
public User(
final String firstName,
final String middleName,
final String lastName,
final String role,
final Date dob,
){
this.firstName = firstName;
this.middleName = middleName;
this.lastName = lastName;
this.role = role;
this.dob = dob;
final version = new Version(this);
this.version = version.version;
this.history.add(version);
return this;
}
}
// Provides a model for version history of a `User` object
@Entity("versions")
class Version {
@Id
private ObjectId id;
private String version;
@Reference
private User data;
public Version(final User user) {
this.data = user;
this.version = this.createVersionId();
}
private String createVersionId() {
ObjectMapper mapper = new ObjectMapper();
// Convert object to JSON string
String json = mapper.writeValueAsString(this.data);
MessageDigest md = MessageDigest.getInstance("SHA-256");
md.update(json.getBytes("UTF-8"));
byte[] digest = md.digest();
String versionString = String.format("%064x", new java.math.BigInteger(1, digest));
return versionString;
}
}
Immutability solves creating consistent backups, not by coordination, but by maintaining a version history of the state of a data object. While this optimizes for recoverability at the last known good state and high availability, the tradeoff is an additional use of disk space, as well as complexity created by maintaining a version history of a data object.
It is near impossible to coordinate database backups according to wall clock time, because of the complexity required to coordinate and synchronize all CPU clocks across multiple servers. Therefore, if the application has access to an array of versions, then the versions of data can be coordinated in the application layer, rather than relying on precise timing to capture state at a precise moment.
A frequently used technique is to use a consistent hashing algorithm with a low collision rate (ie: SHA256
) to create a version identifier based on the contents of the data object. See the example on the right.
Halting Database Writes
If an application uses multiple databases and is not architected to use versioning for data objects, the best way to capture a consistent state across multiple databases is to halt writes to all databases. This can be achieved by planning an outage for an application.
An outage is a less-than-ideal solution if your application requires high-availability. To minimize the impact of halting writes to the databases, the application could be architected to take advantage of message queues (ie: MessageHub) and the micro-services architectural pattern. In this scenario, micro-services are created to accept writes for each database. These applications can be coordinated to stop accepting writes and take a snapshot of the database.
A message queue, such as MessageHub, could be used to fan data objects to the datastore micro-services. In this case, the queue can still accept the objects for writes, while the micro-services are halted for capturing consistent snapshots.
The addition of a caching micro-service, such as Redis, may allow your application to have a read-your-write consistency model for some services; however services requiring backup, such as an Elasticsearch full-text search service, would not have the latest dataset to query until writes to the service were restored.
Replication and Failover
Many of the open source databases vendors may provide replication products which are unsupported in Compose.
Restore
Compose’s user interface (UI) provides a method to to restore databases from On-Demand or Scheduled backups. Restoring a database through Compose’s UI is straight-forward, but does warrant a discussion with regards to best practices. Detailed below will are topics that should be understood before starting a backup and restore process.
How Restore Works
To understand how to restore, it is important to understand how backups are designed. In short, for most deployments, backups are taken from a hidden cluster member, in which writes are halted and then the database files are archived from the filesystem into an object store.
In Compose, restoring from a backup creates a new deployment identical to the previous deployment. Compose will pull the data from the remote object store, place the archive files onto the filesystem, and then start the new database processes in the cluster. This restores the database to the point in time from when the backup was created.
Only etcd
and rethinkdb
are not restored with the original database files. In these databases, etcdtool
and rethink restore
are used, respectively. This is an implementation detail and does not affect how users make use of Compose’s restore functionality.
Compose Restore Instructions
The following instructions are how to restore a deployment for a Bluemix dedicated application, generalized for all types of Compose database deployments.
- Login to Bluemix
- On the Bluemix left-hand navigation, click the Services link and a specific Compose Service deployment.
3. From the Compose service dashboard in Bluemix click the Launch button to get through the Compose dashboard.
4. Select the deployment to access the Overview page, for example a MongoDB deployment
5. From the Overview dashboard, select the Backups
to view the archive of Scheduled and On-Demand backups.
6. From the Backups view, there are two options, either download or restore. In order to restore to a new deployment, click the Restore icon.
7. On the Restore view, deployment specific options (such as SSL or storage engines) will be displayed, as well as the target datacenter. Unless the location of the Bluemix application has changed, ensure the proper target datacenter is chosen.
8. Once the restore process is initiated, you will see the following notification. The time for the process to provision an new deployment is a function of the size of the previous database; however, because it is being restored from the database files from the filesystem, the restore process is generally quick.
9. Once the provisioning of the new deployment is complete, then go to the Overview dashboard and scroll to the connection information.
- With the new deployment connection string, update the user-provided services for the application.
Considerations
Authentication and Connections
The most important consideration when restoring a database, is how the application creates the connection - either through VCAP_SERVICES
or user-defined environment variables. When a Compose service is created and bound to an application, many template applications make use of the default connection information in VCAP_SERVICES
. This becomes problematic when using the Compose database restore functionality, because of how restore works by creating a new database deployment.
If the application makes use of default VCAP_SERVICES
, then it is the best practice is to change the application to use user-provided service binding. See User Provided Service Binding for an example.
Manually restoring a database should be avoided. See the discussion below for more details.
Coordinating Restore over Multiple Database Deployments
The Compose Restore functionality can be thought of as a “duplicate deployment” functionality, as opposed to “restore” to a running database deployment. Compose restores Using the Compose Restore UI, it is not possible to restore multiple databases at once. Additionally, the Compose API has been deprecated for use with any deployment. It is only available for legacy deployments of a “Classic MongoDB” deployment.
Restoring Complete Compose Cluster (all deployments)
As an administrator, it is not possible to restore all Compose databases for a given Bluemix Dedicated Cloud. Database restores need to be executed on a per-application basis. This role and responsibility is the application owners in a Bluemix Dedicated Cloud. There is no UI or API available to cloud administrators to restore all databases.
Manually restoring a database should be avoided. See the discussion below for more details.
Failure
If a restore fails, the new database deployment will provide notice and the connection strings will not be available on the Overview dashboard. If a failure occurs, the Compose team will be notified, however it is advised to submit a support ticket.
Manual Restore Instructions
A manual database restoration is best suited for local development environments or a migration from a database deployment that is not in Compose.
MongoDB
Using A Compose MongoDB Backup
Compose MongoDB backup files are not created with mongodump
, instead they snapshots of the database files. See Backup Design for more information.
- Download a backup from a MongoDB deployment from Compose
- Decompress the tar archive
- Start MongoDB locally with the database files:
monogod --dbpath /path/to/directory
Using a Manual Backup
For a Compose deployment
While mongorestore
is a well documented tool for Mongo database restoration, for production environments in Compose the best practice is to follow the instructions for Compose Restore. If an application makes use of the VCAP_SERVICES
connection strings, it may also make sense to create a new Compose service binding to your application and then use this method to restore the database.
- Login to Bluemix
- On the Bluemix left-hand navigation, click the Services link and a specific Compose Service deployment.
3. From the Compose service dashboard in Bluemix click the Launch button to get through the Compose dashboard.
4. From the Compose service, select the target MongoDB deployment Overview dashboard, and obtain the connection information.
5. Execute the following command to restore to Compose-hosted MongoDB deployment:
mongorestore --ssl --sslAllowInvalidCertificates -h sl-us-dal-9-portal.1.dblayer.com -p 10233 -d DBNAME -u USERNAME -pPASSWORD
More about mongorestore
can be found on the MongoDB documentation
Postgres
Using A Compose Postgres Backup
Compose Postgres backup files are not created with pg_dump
, instead they pg_basebackup
, which produces a set of data files which can be used to start Postgres. See the Compose documentation and the pg_basebackup
documentation for more information.
To install a Compose Backup on your local Postgres installation:
- Download a backup from a Postgres deployment from Compose
- Decompress the tar archive
- Follow the instructions for a Point-In-Time-Recovery
Using a Manual Backup
For a local Postgres installation:
For a Compose deployment
While pg_restore
is a well documented tool for Postgres database restoration, for production environments in Compose the best practice is to follow the instructions for Compose Restore. If an application makes use of the VCAP_SERVICES
connection strings, it may also make sense to create a new Compose service binding to your application and then use this method to restore the database.
- Login to Bluemix
- On the Bluemix left-hand navigation, click the Services link and a specific Compose Service deployment.
3. From the Compose service dashboard in Bluemix click the Launch button to get through the Compose dashboard.
4. From the Compose service, select the target Postgres deployment Overview dashboard, and obtain the connection information.
5. Use pg_restore
with the connection strings provided by Compose. For example:
pg_restore -d DBNAME -h <HOST NAME> -p 10109 -U admin /path/to/dump
More about pg_restore
can be found on the Postgres documentation
ElasticSearch
Elasticsearch does not provide a tool for dumping and restoring an Elasticsearch database. However, an tool called elasticdump
was provided by the open-source community. Its use requires that Node and NPM are installed.
Compose does provide the ability to download a backup an Elasticsearch cluster; however following the Compose Restore Instructions, it is possible to restore an Elasticsearch database in Compose.
Using a Manual Backup
For a local Elasticsearch installation:
See the Elasticsearch manual backup section for instructions on how to install elasticdump
- Obtain a Manual backup - NOTE: It is possible to read from one Compose Elasticsearch deployment and into a local or remote elasticsearch host.
- Follow the
elasticdump
instructions to restore. For example:
elasticdump --input=path/to/elasticdump.json --output=https://USER:PASSWORD@<HOSTNAME>:10331 --type=data
For a Compose deployment
For production environments in Compose, the best practice is to follow the instructions for Compose Restore.
- Login to Bluemix
- On the Bluemix left-hand navigation, click the Services link and a specific Compose Service deployment.
3. From the Compose service dashboard in Bluemix click the Launch button to get through the Compose dashboard.
4. From the Compose service, select the target Elasticsearch deployment Overview dashboard, and obtain the connection information.
5. Use elasticdump
with the connection strings provided by Compose. For example:
elasticdump --input=https://USER1:PASSWORD1@<HOSTNAME>:10331 --output=https://USER2:PASSWORD2@<HOSTNAME>:10332 --type=data
Redis
Using A Compose Redis Backup
Compose Redis backup files are created with redis-cli
and dump the rdb file. These instructions can be used to create a redis-server
for local development:
- Download a backup from a Redis deployment from Compose
- Decompress the tar archive
- Start Redis locally with the database
- Import the dump file:
<!– TODO: Verify on Client Cloud> –>
redis-server --dbfilename dump.rdb --dir .
Using a Manual Backup
The instructions for a Manual backup are identical to the Compose provided backup file, mention directly above.
For a Compose deployment
Restoring a Compose Redis deployment for production environments in Compose should follow the best practice found in the instructions for Compose Restore. If an application makes use of the VCAP_SERVICES
connection strings, it may also make sense to create a new Compose service binding to your application and then use this method to restore the database.
- Login to Bluemix
- On the Bluemix left-hand navigation, click the Services link and a specific Compose Service deployment.
3. From the Compose service dashboard in Bluemix click the Launch button to get through the Compose dashboard.
4. From the Compose service, select the target MongoDB deployment Overview dashboard, and obtain the connection information.
5. Execute the following command to restore to Compose-hosted Redis deployment:
redis-cli -h <HOSTNAME> -p 14250 -a SAQABRBLMAPXPXFM --rdb dump.rdb
More about redis-cli
can be found on the Redis documentation
RethinkDB
See the RethinkDB backup section for more information.
All RethinkDB backup files are created with rethinkdb dump
into a file. These instructions can be used to recreate a RethinkDB for local development or for a Compose RethinkDB deploymnet.
- Download a backup from a RethinkDB deployment from Compose
- Decompress the tar archive
- Start Rethink locally with the database
- Import the dump file using
rethinkdb restore
<!– TODO: Verify on Client Cloud> –>rethinkdb restore DATAFILE
For a Compose Deployment
- Login to Bluemix
- On the Bluemix left-hand navigation, click the Services link and a specific Compose Service deployment.
3. From the Compose service dashboard in Bluemix click the Launch button to get through the Compose dashboard.
4. From the Compose service, select the target Rethink deployment Overview dashboard, and obtain the connection information.
5. Execute the following command to restore to Compose-hosted RethinkDB deployment:
redis-rethinkdb restore rdb_dump.tar.gz --tls-cert compose.rethinkdb.crt -c <HOSTNAME>:39500
More about redis-cli
can be found on the Redis documentation
Considerations
From Bluemix
It is not advised to store data in a Bluemix application. Doing so will likely require custom buildpacks and require coordination. The only advantage is reducing network latency, however the data would still be required to be pushed to the application instance. In short, avoid if possible.
Performance
The performance of a manual restore will vary greatly, depending on, but not limited to, database size, network latency, database type, etc. When possible, use Compose’s restoration. It will be the fastest path to restoration.
Failure
For any restore, whether the deployment is local or on Compose, the best practice is to restore to a fresh and clean database. Moreover, every effort should be made to restore from Compose, to make use of the best practices and stable and reliable tooling previously implemented.
If a failure is encounter while manually importing a database, please contact support.
Coordinating Restoring Multiple Deployments
Restoring Complete Compose Cluster (all deployments)
As a system administrator, it is not possible to backup all database deployments in a Compose cluster, nor is it possible to restore all databases at once. In a cloud environment, such as Bluemix Dedicated, the onus of restoring databases generally falls in the hands of the application owner.
Disaster Recovery Runbook
This documentation is to assist the IT administrators and application owners administrating applications on an IBM Bluemix Dedicated environment during a unplanned outage of the Compose Service.
Roles and Responsibilities
- Client Application Owners - end-users that are the stakeholders with deployed applications on the Bluemix and Compose platforms. Application owners are responsible for configuring their applications with credentials and hostnames.
- Client IT Administrators - coordinates communication and incident management between the client and IBM’s support teams. Client IT administrators may be required to adjust any egress network connectivity that may be required to reach the cluster(s) as they will have different IP addresses.
- Bluemix Support - provides communication to the client on current status, concerns and issues.
- Compose Support - provides incident response and management for the Compose service. Compose is responsible for standing up hosts, preparing backup data for restore, and loading data as necessary into new database instances deployed on the new cluster.
- Bluemix Operations - provides incident response and management for the Bluemix platform. Responsible for service broker deployment.
- Softlayer Operations - provides infrastructure support to Bluemix Support, Bluemix Operations, and Compose Support.
Disaster Recovery Outline
1. Incident
The Compose service is no longer accessible from the Bluemix Environment. Application monitoring may begin to raise alerts to Client Application Owners and/or Client IT Administrators. Compose receives notifications from monitors about outage.
2. Submit Support Ticket
Client IT Administrators submit a Severity 1 support ticket with the Bluemix Dedicated Support team through the Bluemix administrative interface. The time to create and submit the support ticket will depend largely on the Client IT Administrators.
3. Support Response
The Bluemix Support team will initiate coordinating incident management and response with Compose Support team and Client IT Administrators. The Bluemix Support team will make the best effort to response as soon as possible. The Service Level Objective (SLO) for Severity 1 response times is 1 hour.
4. Outage Identification
The Compose Support and Bluemix Operations team determine the Compose cluster is not fixable within the current data center.
5. Deployment Identification/Confirmation
The Cloud Foundry CLI command to unbind services
$ cf unbind-service APP_NAME COMPOSE_SERVICE_INSTANCE
Or the Bluemix CLI
$ bx service unbind APP_NAME SERVICE_INSTANCE
The Compose Support team determines affected database deployments. The identified deployments are confirmed with the Client IT Administrators. The time objective is 10 minutes.
6. Unbind Compose Services from Impacted Applications
The Bluemix Support team would advise the Client IT Administrators to unbind the existing Compose services from applications. The Client IT Administrators should notify the Application Owners to unbind the Compose service from their respective applications. The time objective should be 10 minutes per application.
7. Disaster Recover Site Infrastructure Deployment
The Compose Support requests that SoftLayer provisions the required server hardware at the new datacenter. The time objective is 1 hour.
8. Compose Services Deployment
The Compose Support team initiates the deployment and configuration of the Compose services on the provisioned hardware. The time objective is 1 hour.
9. Network Configuration
The Compose Support team provides IP addresses and network configuration for the VPN to the Bluemix Operations team. The time objective is 10 minutes.
10. Service Broker Update
The Bluemix Operations team configures a new service broker in the Bluemix Dedicated environment to the new Compose cluster deployed at the new datacenter. The time objective is 30 minutes.
11. Database Deployment Restore
The Compose Support team initiates restore function for database deployments identified in the affected datacenter. Time variable per deployment instance. The recovery time objective for each database may take between 30 minutes and 2 hours to restore depending on data size, index size, and configuration. The recovery point objective is within the last 24 hours, based on the last available backup.
12. Service Broker Confirmation
The Compose Support and Bluemix Operations team verify the updated service broker is properly configured and functional. The time objective is 10 minutes.
13. Service Binding
The Client Application Owners and/or Client IT Administrators create user-provided services in Bluemix for each application. Please see the User Provide Service instructions. The time objective is less than 10 minutes per application.
14. Application Recovery
To restart an application using the Cloud Foundry CLI
$ cf restage APP_NAME
To restart an application using the Bluemix CLI
$ bx app restage APP_NAME
To redeploy an application using the Cloud Foundry CLI
# Add updated Compose SSL certificate to filesystem
$ cf push APP_NAME
To redeploy an application using the Bluemix CLI
# Add updated Compose SSL certificate to filesystem
$ bx app push APP_NAME
The Client Application Owners should restart or redeploy the application based on the application needs. The time objective is less than 10 minutes per application.
- Filesystem: Applications that read the Compose SSL certificate from the filesystem will need to be redeployed with the updated Compose SSL certificate available from the Compose Dashboard.
- Services: Applications that read from environment variables, such as
VCAP_SERVICES
or user-defined environment variables may only require the application to be restaged, in order for the application to re-read the new service credentials.
15. Application Recovery Confirmation
The Client Application Owners and Client IT Administrators verifies impacted applications are now able to respond to production requests. The time objective is less than 10 minutes per application.
Resolution
Migrating from the disaster recovery (DR) site would entail creating (or restoring) the original cluster to a known-good state, followed by a full backup of the DR site being taken and then applied on the original cluster.
The Compose and Bluemix Support teams are primarily responsible for coordinating the restoration and migration of the original datacenter site. The Compose and Bluemix support team will coordinate directly with the Client IT administrators.
Escalation Path
Client IT Administrators should contact their IBM account managers if an escalation is required.
Applicable Situations
This runbook was designed to address a complete loss of the Compose service from the Bluemix platform. Applicable situations include, but are not limited to:
- Natural Disaster
- Datacenter power loss
- Irreparable loss of network connectivity
User Provided Service Binding
The examples below are for application owners and administrators. Each database in Compose has slightly different authentication scheme; however the schema of the JSON document for credentials in VCAP_SERVICES
is arbitrary. Ultimately, the key/value pairs in a user-provided service are for the user to define and only relevant to the application that consumes the VCAP_SERVICES
environment variables.
Below are examples for how to create and bind a user-provided service to an existing application. These directions can be used with any database deployment in Compose. Either the Cloud Foundry or Bluemix CLI is required, examples will be given for each.
1. Get DB credentials
Login to the Compose Dashboard, select the target MongoDB deployment, and on the Overview page, scroll down to the Connection Info
section and obtain the appropriate credentials and SSL certificate.
2. Create Credentials JSON
credentials.json
{
"db_type": "mongodb",
"db_name": "<DB_NAME>",
"db_user": "<DB_USER>",
"db_pass": "<DB_PASS>",
"ca_certificate_base64": "[REDACTED]",
"uri": "mongodb://<USER>:<PASSWORD>@bluemix-sandbox-dal-9-portal.1.dblayer.com:15460/<DB_NAME>?ssl=true"
}
Create a valid JSON document with the appropriate key/value pairs for the credentials, based on how the application is configured to consume VCAP_SERVICES
. Save the json document to your local filesystem.
3. Create Service
Create a user-provided service with the Cloud Foundry CLI:
$ cf cups my-custom-service -p credentials.json
Create a user-provided service with the Bluemix CLI:
$ bx service user-provided-create my-custom-service -p credentials.json
Using the cf
or bx
tool, create a new user-provided service.
4. Bind Service
Bind a user-provided service with the Cloud Foundry CLI
$ cf bs my-app my-custom-service
Bind a user-provided service with the Bluemix CLI
$ bx service bind my-app my-custom-service
Bind the newly created user-provided service to the application
5. Restage
Restage the application with the Cloud Foundry CLI
$ cf restage my-app
Restage the application with the Bluemix CLI
$ bx app restage my-app
Finally, simply restage
the application
RTO and RPO
The following documents contain information about RTO and RPO for the Compose services:
| Compose Service | URL |
|---|---|
| MongoDB | MongoDB RTO |
| Postgres | Postgres RTO |
| Elastic Search | Elastic Serch RTO |
| Redis | Redis RTO |
Cloudant
Cloudant is not a traditional database, but rather a Database-as-a-Service (DBaaS). Moreover, Cloudant is not a relational database (aka RDBMS), but rather is a “NoSQL datastore”. As a NoSQL DBaaS, Cloudant is a new paradigm when compared to a RDBMS, does not provide the ability to do backups.
Cloudant’s design and architecture optimize for durability and availability of data, large datasets, and operational simplicity. With regards to “operational simplicity”, Cloudant is designed to be resilient when facing server outages. Moreover, Cloudant makes use of three properties which insure the durability and availability of data:
- Internal replication: data records are replicated to three different machines, ensuring the data is secure and available
- Multi-Version Concurrency Control: the causality of data mutation (ie:
UPDATE
) is deterministic and version conflicts are retained, and presented to the end user for resolution. - Delete Tombstones:
DELETE
operations on a data document create an incremented version of the data, with no data in the current version.
The above are properties of an Eventually Consistent database. With an eventually consistent design, traditional backups are not necessary because of the internal replication. In an eventually consistent system, like Cloudant, there are three backups of the data at all times.
Eventual consistency does provide better guarantees of data durability and availability in a single datacenter than a RDBMS. With that said, in the event of failure of the whole datacenter, the best practice advises to use replication for backups and disaster recovery.
Eventual Consistency
At the core of Cloudant is an open-source database called CouchDB. Cloudant has extended CouchDB by adding many features, ranging from geospatial indexing to full-text search. Moreover, Cloudant made CouchDB an eventually consistent database service.
Eventual Consistency describes the consistency model for data or state in a distributed system. In Cloudant’s case, when the database service is provisioned, a cluster of servers (a.k.a.: “nodes”) are tasked to store data and respond to read/write requests. Cloudant is a distributed system.
In a distributed system, a user wants three guarantees of the data - consistency, availability, and partition tolerance. Partition tolerance refers to the ability of the system to continue to operate, despite network or host failure. Consistency implies that every host responding to a query for data will provide the identical response. Availability means the data is there when it is requested. However in failure states or concurrent operations, in a distributed system, the user must choose between consistency or availability of the data.
Eventual consistency is a stark difference to traditional RDBMS. RDBMSs’ are generally configured to have master-slave replication. If the master node is unavailable, then writes are unavailable and consistency is preserved. In Cloudant, if a node fails, any other node can accept a write, thus the data remains available for write operations.
In failure cases or concurrent operations, Cloudant is optimized for availability, whereas traditional RDBMSs are optimized for consistency. Cloudant’s internal replication is near real-time (bounded by network latency), so inconsistency is an exception, not the rule. Inconsistencies arise during node failures in a cluster, or during race conditions (aka concurrent operations). Inconsistencies are resolved by the use of Cloudant’s Multi-Version Concurrency Control. Moreover, as a Database-as-a-Service, node failure in a Cloudant cluster is not exposed to the end-users of Cloudant.
Internal Replication
Eventual consistency implies that the database system maintains multiple copies of data. When data is sent to a Cloudant cluster, three (3) nodes in the cluster write the data to disk. Creating three copies of the data ensures the data remains highly available, even when one or more nodes in the cluster are unavailable.
When compared to a traditional RDBMS which may only store one copy of the latest version data, Cloudant stores 3 copies of the most recent versions of the data. By storing 3 versioned copies, this is analogous of maintaining two real-time backups of a traditional RDBMS.
A master-slave configuration of an RDBMS may achieve a similar assurance of having numerous copies of the data; however, it is ultimately limited in scale. Cloudant can scale data partitioning or operation effortlessly, whereas an RDBMS would become bounded or complex.
Multi-Version Concurrency Control
first revision of the document
{
"_id": "74b2be56045bed0c8c9d24b939000dbe",
"_rev": "1-7438df87b632b312c53a08361a7c3299",
"name": "Samsung Galaxy S4",
"description": "",
"price": 650
}
second revision of the document
{
"_id": "74b2be56045bed0c8c9d24b939000dbe",
"_rev": "2-61ae00e029d4f5edd2981841243ded13",
"name": "Samsung Galaxy S4",
"description": "Latest smartphone from Samsung",
"price": 650
}
also second revision, conflicts with the previous one
{
"_id": "74b2be56045bed0c8c9d24b939000dbe",
"_rev": "2-f796915a291b37254f6df8f6f3389121",
"name": "Samsung Galaxy S4",
"description": "",
"price": 600
}
The internal replication uses a versioning tracking system called “Multi-Version Concurrency Control”(aka MVCC) to track the changes in the data. This means that when a record is updated/mutated, the previous data is not removed until two conditions are met:
- All nodes hosting the data agree about the latest version, there are no conflicts.
- The user configurable compaction process identifies old, stale versions of the data to be removed.
In a traditional RDBMS, an UPDATE
operation mutates the data record and the previous values are discarded. In Cloudant, the previous values are stored for a period of time, until a compaction process occurs.
MVCC assigning a version number and an identifier of the node which incremented version number. This makes the causality of the data mutation is deterministic. When failures or concurrent write operations occur, then the multiple versions in conflict are presented to the application layer (or end-user) for resolution.
By versioning data, the potential for data loss is mitigated, because Cloudant ensures the latest data has been replicated to three other hosts. If there is a conflict from a failure or concurrent mutation, no data is lost, and the choice is given to the end-user on how to resolve conflicts.
Delete Tombstones
Data in Cloudant is not physically removed until a time and manner determined by an administrator, not an end-user or application. From the perspective of an application, when a DELETE
operation is executed, the data document is removed. From the internal Cloudant perspective, the data record’s version is incremented per MVCC and marked as deleted. This is called a Tombstone Document.
Tombstone documents are not deleted. The previous versions of the document will remain in the database until the compaction process occurs. The tombstone document is ultimately used for replication purposes, to ensure the deleted version of the document is circulated to the other hosts of the data.
Tombstones documents make use of MVCC. Unlike RDBMS, data is deleted immediately and unrecoverable once deleted, unless restoring from a backup. Cloudant provides a window in which data is immediately recoverable, based on the compaction schedule.
The timeframe until the compaction process occurs is based on the throughput of the system. Compaction can be adjusted in priority for Cloudant dedicated customers. To adjust the compaction schedule, users with the cluster admin
role must create a support request with the Cloudant support team.
An example of a filter function to remove tombstone documents from a replication stream:
Backup and Replication
Cloudant’s replication provides the facility to create a database backup, and store it on a Cloudant cluster. From a replicated cluster, it is possible to restore a specific data document or an entire database.
Cloudant has been designed (eventually consistent, highly-available) so that backups for data redundancy is no longer necessary. Backups are still a good practice, but it is important to understand the purpose of the backup, in order to choose a backup strategy.
For example, if an application requires low latency, replication can be used multi-master configuration. If low latency is not a concern, but application data must be highly available, then replication can be used for a warm-standby configuration. If an application needs to meet regulatory requirements, replication can be used for point-in-time or incremental backups; however if a history of data documents needs to be stored for compliance purposes, it may be advantageous to use architect/design the application to store versions of documents.
In summary, it is important to consider the purpose of the backup in order to choose the strategy that will best meet the objective of the backup. Due to Cloudant’s fundamental design choices, it is highly unlikely that a backup will be needed to recover an entire cluster. It is highly likely that the use case of your application will determine the backup strategy required.
Full Backups and Point-in-Time
Full and point-in-time backups are trivial to create with replication. The method is to use Cloudant replication from the primary cluster to a secondary cluster. Once replication is complete, disable replication, note the point-in-time the database was created.
To create multiple point-in-time backups on a schedule, such as seven daily and four weekly, this would require significant disk space, especially if the dataset is large.
Incremental backups
To create an incremental backup, a high-level description of the process is as follows:
- Perform an initial backup of the entire database by using replication.
- Schedule a job locally to backup the differences, or deltas, from the previous 24 hour period.
Creating an incremental backup
Incremental backups save only the differences or ‘deltas’ between backups. Every 24 hours, the source database is replicated to a target database.
Replication uses sequence values to identify the documents changed during the 24-hour period. The backup operation works by using replication to get and store a checkpoint.
The backup operation creates the name from a combination of the date and the backup task name. This name makes it easier to identify checkpoints during the recovery or roll up process.
To create an incremental backup, perform the following steps: <!– TODO: Verify on Client Cluster –>
- Find the ID of the checkpoint document for the last replication. It is stored in the
_replication_id
field of the replication document, found in the_replicator
database. - Open the checkpoint document at
/<database>/_local/<_replication_id>
, where<_replication_id>
is the ID you found in the previous step, and<database>
is the name of the source or the target database. The document usually exists on both databases but might only exist on one. - Search for the
recorded_seq
field of the first element in the history array found in the checkpoint document. - Start replicating to the new incremental backup database, setting the
since_seq
field in the replication document to the value of therecorded_seq
field found in the previous step.
Restoring a database
To restore a database from incremental backups, replicate each incremental backup to a new database, starting with the most recent increment. While it may be intuitive to start restoration from the oldest backup, this is not advised. Starting replication from the newest/latest incremental backup first is faster because updated documents are only written to the target database once. Any documents older than a copy already present in the new database are skipped.
An example
This example shows how to:
- Setup databases to use incremental backup.
- Run a full backup.
- Set up and run an incremental backup.
- Restore a backup.
Constants used in this guide
# save base URL and the content type in shell variables
$<EMAIL_ADDRESS>$ ct='Content-Type: application-json'
In this example, the following assumptions are made:
- Only one backup is made for only one database;
- A full backup will be created on Monday
- An incremental backup will be created on Tuesday
curl
and jq
commands to run these operations. Other excellent tools are http-prompt
and HTTPie.
Step 1: Check you have three databases
Check you have three databases to use with this example
$ curl -X PUT "${url}/original"
$ curl -X PUT "${url}/backup-monday"
$ curl -X PUT "${url}/backup-tuesday"
For this example, three databases are required:
- The original database, holding the canonical data to backup.
- Two incremental databases, for Monday (
backup-monday
) and Tuesday (backup-tuesday
).
Step 2: Create the _replicator
database
Create the
_replicator
database
$ curl -X PUT "${url}/_replicator"
If it does not exist, create the _replicator
database.
Step 3: Back up the entire (original) database
Run a full backup on Monday
$ curl -X PUT "${url}/_replicator/full-backup-monday" -H "$ct" -d @backup-monday.json
backup-monday.json
must contain the following:
{
"_id": "full-backup-monday",
"source": "${url}/original",
"target": "${url}/backup-monday"
}
On Monday, back up all data for the first time by using replication from the source original
to target backup-monday
.
Step 4: Get checkpoint ID
Get checkpoint ID to help find the
recorded_seq
value:
$ replication_id=$(curl "${url}/_replicator/backup-monday" | jq -r '._replication_id')
On Tuesday do an incremental backup, rather than another full backup.
To start the incremental backup, two values are needed:
- The checkpoint ID.
- The
recorded_seq
value.
These values identify where the last backup ended, and determine where to start the next incremental backup. With these values it is possible to run the incremental backup.
Obtain the checkpoint ID value, which is stored in the _replication_id
field of the replication document in the _replicator
database.
Step 5: Get recorded_seq
value
Get
recorded_seq
from original database
$ recorded_seq=$(curl "${url}/original/_local/${replication_id}" | jq -r '.history[0].recorded_seq')
After obtaining the checkpoint ID, use the ID to get the recorded_seq
value. This is found in the first element of the history array in the /_local/${replication_id}
document, within the original database.
The last document replicated from the original database can be determined by the recorded_seq
value.
Step 6: Run an incremental backup
Start Tuesday’s incremental backup
$ curl -X PUT "${url}/_replicator/incr-backup-tuesday" -H "${ct}" -d @backup-tuesday.json
where backup-tuesday.json contains the following:
{
"_id": "incr-backup-tuesday",
"source": "${url}/original",
"target": "${url}/backup-tuesday",
"since_seq": "${recorded_seq}"
}
With the checkpoint ID and recorded_seq
, the Tuesday’s incremental backup can be initiated. This replicates all the document changes made since the last replication.
Once the replication finishes, an incremental backup will be complete. The backup consists of all the documents in the original database, and may be restored by retrieving the content of both the backup-monday
and backup-tuesday
databases.
Step 7: Restore the Monday backup
Restore from the
backup-monday
database
$ curl -X PUT "${url}/_replicator/restore-monday" -H "$ct" -d @restore-monday.json
# where restore-monday.json contains the following:
{
"_id": "restore-monday",
"source": "${url}/backup-monday",
"target": "${url}/restore",
"create-target": true
}
To restore from a backup, replicate the initial full backup, and any incremental backups to a new database.
For example, to restore Monday’s state, replicate from the backup-monday
database.
Step 8: Restore the Tuesday backup
Restore Tuesday’s backup to get the latest changes first
$ curl -X PUT "${url}/_replicator/restore-tuesday" -H "$ct" -d @restore-tuesday.json
restore-tuesday.json
contains the following json document:
{
"_id": "restore-tuesday",
"source": "${url}/backup-tuesday",
"target": "${url}/restore",
"create-target": true
}
Finish by restoring Monday’s backup last
$ curl -X PUT "${url}/_replicator/restore-monday" -H "$ct" -d @restore-monday.json
restore-monday.json
contains the following json document:
{
"_id": "restore-monday",
"source": "${url}/backup-monday",
"target": "${url}/restore"
}
To restore Tuesday’s database, first replicate from backup-tuesday
and then from backup-monday
.
It is possible to restore in chronological sequence; however, by using the reverse order, documents updated on Tuesday only need to be written to the target database once. Older versions of the document stored in the Monday database are ignored.
Best practices
While the previous information outlines the basic incremental backup process, each application has its own requirements and strategies for backups. Here are a few best practices you might want to keep in mind.
Scheduling backups
Replication jobs can significantly increase the load on a cluster. If backing up several databases, it is best to stagger the replication jobs for different times, or to a time when the cluster is less busy.
Changing the IO priority of a backup
Setting the IO priority
X PUT "${url}/_replicator/io_priority" -H "$ct" -d @io_priority.json
io_priority.json
contains the following json document:
{
"source": {
"url"<EMAIL_ADDRESS>"headers": {
"x-cloudant-io-priority": "low"
}
},
"target": {
"url"<EMAIL_ADDRESS>"headers": {
"x-cloudant-io-priority": "low"
}
}
}
The response body will look like:
{
"ok":true,
"id":"io_priority",
"rev":"1-c3bc51284a2cc0ea0167178d23a8f553"
}
You can change the priority of backup jobs by adjusting the value of the x-cloudant-io-priority
field within the replication document.
- In the source and target fields, change the
headers
object. - In the headers object, change the
x-cloudant-io-priority
field value to"low"
.
Backing up design documents
Example Filter Function to remove Design Documents
function(doc, req) {
if (doc._id.indexOf('_design') != 0) {
return false;
}
return true;
}
If you backup design documents, the backup operation creates indexes on the backup destination. This practice slows down the backup process and uses unnecessary amounts of disk space. If you don’t require indexes on your backup system, use a filter function with your replications to filter out design documents. You can also use this filter function to filter out other documents that are not required.
Backing up multiple databases
If your application uses one database per user, or allows each user to create several databases, you need to create a backup job for each new database.
Local backups
Cloudant provides a full-featured software version of the Cloudant service, called Cloudant Local. Cloudant Local can be a source or target for replication and is configured in the same manner as the service-based Cloudant replication.. Proper configuration requires that Cloudant Local is connected to and is accessible to a network and has a route to and from the Cloudant service.
More information about Cloudant Local can be found at the Cloudant Local documentation site.
Security Considerations
In order for replication to proceed optimally when replicating from database “a” to database “b”, the credentials supplied must have:
-
_reader
and_replicator
rights on database “a”. -
_writer
rights on database “b”.
API keys are generated in the Cloudant Dashboard or through the API.
Each key can be given individual rights relating to a specific Cloudant database. Cloudant must be able to write its checkpoint documents at the “read” end of replication, otherwise no state is saved and replication cannot resume from where it stopped. If the state is not saved, it can lead to performance problems when resuming replications of large data sets. The reason is that without checkpoints, the replication process restarts from the beginning each time it is resumed.
Admin access is required to insert a document into the _replicator
database. The login credentials supplied in the source and target parameters do not require full admin rights. It is sufficient if the credentials are able to:
- Write documents at the destination end.
- Write checkpoint documents at both ends.
Cloudant has a special _replicator
user permission. This allows checkpoint documents to be created, but does not allow the creation of ordinary documents in a database. It is recommended that you create API keys that have:
_reader
and_replicator
access at the source side._writer
access at the destination side.
RTO and RPO
The following document contains information about RTO and RPO for the Cloudant: Cloudant RTO
Message Hub
Message Hub is a scalable, distributed, high throughput message bus on the IBM Cloud, available as a fully managed Bluemix service. It is based on Apache Kafka - a fast, scalable, and durable real-time messaging engine from the Apache Software Foundation. To understand the benefits of using Apache Kafka as a service, see Message Hub: Apache Kafka as a Service.
Kafka, and subsequently Message Hub, was designed as a fault-tolerant, distributed system, so Message Hub should remain highly available even during failure conditions. At this time, Message Hub, does not provide the ability to backup or restore message queues or queue metadata. If a catastrophic failure were to occur, applications and application owners will have to re-establish the necessary queues and possibly employ “retry logic” to ensure a message is sent.


About Us
Education
| BTE D3110576: | Breeders and Owners Anne Pounds & Liam Philpott |
| Breeding Book Prefix: | Purebred Horned Herefords : Allowdale |
| Address: | Allowdale Herefords, Knocknacolan, Kanturk, Co. Cork, P51 F677 |
| SNP Genotyping: |
1. Since 2014 2. All cattle bred by Breeders and Owners Anne Pounds & Liam Philpott are SNP Genotyped |
Experience
| WHPR: | Whole Herd Performance Recording |
| BDGP 2015-2020: | Beef Data Genomics Programme (EU Funded) |
| BEEP: | Beef Environmental Efficiency Pilot (EU Funded) |
| GREENBREED Project: | EU Funded |
Skills
Performance
Environmental
Genomics
Programme
New 2021 Hereford Breeding Programme
A new Purebred EU Hereford "Breeding Programme" is being established in accordance with the EU "Animal Breeding Regulation" 2016/1012
The goals of the new purebred EU Hereford "Breeding Programme" is to make use of the technology and science available:
- To provide a higher level of transparency to breeders with respect to their breeding animals;
- To provide a longer notification window to decide on the suitability of an animal for purebred breeding; and
- To enable breeders to have a voice in accordance with the regulation in order to shape the breeding programme.
Links
Register Interest
Please provide your details:
Address
Knocknacolan
Kanturk
County Cork
Ireland
P51 F677
Phone
+353 87 613 6750


{# {% extends "sidebar-page.html" %} #} {% extends "base.html" %} {% block content %}
A number of institutions are independently committing substantial funding for DOAJ thereby enabling DOAJ to become more self-sustaining. Others have committed to sustainable funding using the model as recommended by the SCOSS coalition. The details of all funding organisations have been published in our Funders spreadsheet.
SCOSS (The Global Sustainability Coalition for Open Science Services) recommended to their member institutions to support DOAJ with larger and dedicated amounts of funding to enable DOAJ to move away from its incremental annual contribution system. Under the new mode,l organisations will work towards sustaining DOAJ for 3 years, giving it more stability for the mid-term and enabling DOAJ to develop a fully comprehensive, longer term development plan for its systems and services.
{% endblock %}